Migration strategies - Amazon Web Services (AWS) Tutorial

- So up to this point in the course, we have looked at fundamental understandings of databases. We've looked at the different AWS services for databases. We've planned and designed our databases. And then we've created some databases. All right, well, what if you don't want to create a database from scratch? What if you already have a database in your organization and you want to move that database into AWS? That's what this chapter is all about. Different ways we can think about and accomplish migrating databases from our local network, into the AWS Cloud. Now, in order to do this, you need some good strategies. And the first thing you need to know is what you have. You've got to look at your existing systems, figure out what they are, and then decide which ones need to be migrated to the cloud and how they might be able to be migrated to the cloud, but it all starts with understanding what you have. So you want to discover your database details and there are different parameters you want to know about your databases. You need to know your database management system type. Is it MySQL, SQL Server, PostgreSQL? What is the DBMS you're using? Because that'll help you select a target in AWS. You need to know the DBMS version. Now this could be a real issue. If you've got a really old database, like say Microsoft SQL Server seven, eight, nine, 10, some of those early versions of SQL Server. And now you want to move it into the cloud. Well, you're not going to have support for those old versions in the cloud because AWS is only going to support production supported versions of Microsoft SQL Server. So not only you have to get it into the cloud, but you've also got to upgrade it at the same time, you have to think about those things. Then you have to think about the database size, structure, and access methods. So how big is the database? Because this helps you to understand what the tasks really like to migrated to the cloud. I mean, if this is a one gigabyte database, that's one thing. If it's 100 gigabytes, it's another. And what if it's several terabytes, now it's going to be more of a challenge thinking about the time window it will take to upload that to the AWS Cloud. And then you have to think about the structure. How's it organized? How's it structured? And is that structure supported in AWS? And then the access methods, how are users currently authenticating to your database? And can we mirror that in some way in AWS or do we have to restructure our applications in some way so that the users will be able to access it once it's in AWS? The next thing to look at with knowing what you have is your usage details. You need to discover how it's being used. How many users are using it? How many transactions are being executed per user? What is the size of the average transaction? And what large workloads do you have? So what I mean by that last one is, do you have any large scale reports that are run on a regular basis? You know, like maybe every Thursday or Friday, you have a paycheck run and it's an intensive report because you have 30,000 employees and all of those checks have to be cut. Or maybe you have a weekly report that is generated for management that looks at hundreds of megabytes of data, or even terabytes of data, extrapolates out information, does analysis, and generates some kind of visual report. So you need to know about those large database workloads and therefore think about how that's going to impact your deployment in AWS. So it all starts with knowing what you have. Next you need to know what you need. So knowing what you have will determine what you need in AWS. For example, if you do have those massive workloads, and now they're going to pull across the cloud, you need to think about your internet connection speed. What kind of bandwidth do I need to accomplish this? Maybe you're a medium-sized business. And right now you've got just a hundred megabit connection to the internet or something like that. If you're going to start moving all of your applications into the cloud, that may not be a sufficient connection anymore. And so you've got to think about your internet connection speeds and make sure you're capable of running everything from the cloud, because you have a fast enough internet connection. You also need to look at the AWS database service. Yes, you might be able to do everything for your databases in RDS. It might take care of everything you need, but what if you are in that situation where you have a really odd database system and you want to move it to the cloud, and there's nothing in AWS that functions like this database system, it's simply not supported. Then you might have to launch an EC2 instance, and then install your database service on that in the cloud in order to get your database to the cloud, because none of the managed services provide what you need. And then you need to know your database instance details. So look at what you're running right now onsite. You're probably going to need something with similar power or capability in the cloud, unless it's just way under utilized. Maybe you have a really powerful server and it's only 10% utilized. Then you can probably get by with less in the cloud than what you had in the hardware. Because remember you can always upgrade your instance type or class at a later time. So you can start with what you need and upgrade it later because you remember you pay for what you configure, not for what you need. So you want to make sure that you only configure what you actually need. And then finally, your authentication methods. You need to determine those, based on users and application access methods. So think about how you're doing it now. Can you merge to an identity and access management model that allows you to create user accounts in the cloud and then they get into the database? Or do you need to simply reconfigure your applications so that all of them log on as the same user, which kind of takes away tracking capabilities unless you track it internal to the database in some way? So you need to think through these issues with moving that application into the cloud. Alright, what I want to do before I end this discussion of migration strategies, is give you a very brief overview of a tool that AWS has released in the last couple of years that really help us get these databases into the cloud. It's called the AWS Database Migration Service. And you will notice that it lets you migrate your database to AWS with minimal downtime and more than 200,000 databases have been migrated. So that tells you, it seems to work. 200,000 databases have already gone through it. And there is an introduction here. You can look at it, if you desire to in your own time. The URL is simply aws.amazon.com/dms for Database Migration Service. Now, what does this thing actually do? Well, what AWS Database Migration Service does is it connects to your database and then copies it up to the AWS Cloud. That can be all it does. So in other words, if you have a MySQL database and it's the same version is what can be supported in the cloud, then it can literally just copy your database up to the cloud for you. The cool thing is you can leave the database online while it's doing it. And then think about this. So let's take Microsoft SQL Server, for an example. Microsoft SQL Server and most relational databases do this. It has what's called a transaction log. So as changes are being made to the database, they're written to the transaction log first and then to the database. So the cool thing about DMS is it will actually copy the database up and then it will go to the transaction logs on your SQL server and automatically apply any transactions that have happened since the copy process started. And in addition, it can keep on updating through the transaction logs until you're ready to flip the switch and say goodbye to the on premises and only go over to the cloud. So it helps you not only migrate it, but migrate it right up to the minute that you flip that switch. Another cool feature of it. You'll see right here at the top, the AWS Schema Conversion Tool. Now really the Schema Conversion Tool is kind of a separate tool, but notice what you can do with it. You have a source Oracle database, you can convert the Oracle schema over to Amazon Aurora, MySQL, PostgreSQL, or just keep it Oracle. You have a Microsoft SQL Server, same thing, go over to Aurora, maybe Redshift, MySQL or PostgreSQL. The point is the Schema Conversion Tool just does just that. It allows you to take the database system you have, and literally convert it to another database system on the fly. Now, of course you'll want to test it with your application to make sure everything works after the conversion, but this is something that can be done, and it integrates with the Database Migration Service. So what I mean by that is the Database Migration Service can migrate the database, and at the same time, use the Schema Conversion Tool to convert it to a completely different kind of database. Maybe you're in MySQL today and for some reason you want to go ahead and convert to Aurora, you can do that in the process automatically.