Steven Lott provides a summary of the changes that have been made in Python 3. This helps clarify the scope of the migration effort. It also provides some historical context and shows the level of technical risk involved. This section identifies the tools and shows how the tools are related to each other.
- [Lecturer] I want to start by looking at the state of Python 3 today and what's involved in migrating programs from Python 2 to Python 3. What's important is that the technology is now well established, it doesn't involve risk, it's really just a matter of a little hard work, with a nice payout. Python 3 has been around for several years. Why was it needed? Python 2 had evolved organically since it was invented, some features which seemed like a good idea at the time didn't age well, the better ideas had accumulated to the point where something had to be done to implement them, the internals of Python had to be changed to make it simpler and faster.
These internal changes lead to three kinds of visible changes in the language. The syntax of a few Python statements has changed, we'll look at some specifics in part four and we'll look at details in chapter two. The builtin functions also needed to be changed, some of the changes are implementation details, some of the changes involve removing some functions that simply were not a good idea, and the standard library also needed rework.
Of course, it needed to be changed to work with the new internals and the new syntax and builtins, it also needed some reorganization to reflect the current state of the art. And before digging into the differences, I'd like to provide an overview of what's involved in migrating. One of the reasons we use Python is because the ecosystem of libraries and tools is vast, there's a lot of software out there, we can install it using pip and use it with an import statement.
The initial concept for migration was a tool called 2to3, which could automate the conversion from old Python 2 to new Python 3, and this served as a bridge into the newer language. With some care to address ambiguous cases it could automate everything. Additionally, the __future package was created, with a simple import, the runtime changed to support the new features. This makes it easy to start using Python 3 features even when writing Python 2 code.
For example, the from__future__import print_function will remove the old print statement and add the new print_function. It creates a syntax change, localized to a script or module. By 2010, the six Package was published. The idea behind six, two times three, clever, right? Is to provide a library that allows a single code base to run in either environment, and this works because the syntax is so similar.
In a way, this is a logical extension of the __future package, it means you don't need to rely on using the 2to3 tool to convert, you can write code to run in both environments, using six. This is a kind of simplification because you have only one code base, but writing with six is a little awkward, and I'll show detailed examples in the next chapter. The python-future Project expands on the capabilities of six, it also shifts the focus slightly.
We can use the future tool set to do a very clean conversion from Python 2 to Python 3. It uses a two-step process that makes it easy to locate the edge cases that can cause confusion. In particular, there are ambiguities around Unicode strings and bytes in Python 2 that future can help identify. This project has a compatible idioms webpage that can help guide you through the issues that will arise in making the conversion.
By 2014 Python 3.4 arrived, and for me, this was the start of converting to Python 3. This release of Python included many of the important frameworks and libraries, in some cases, where a project was not keeping up, replacements had been built. Since the foundations for using Python 3 have been available for a while now, it's time to convert remaining applications to leverage all the new features, and while it's not official, many of us are looking at PyCon 2020 as being the end of life for Python 2.7.
Check the pythonclock.org website to see a sense of urgency here. There are still some gaps between the ideal world of Python 3 everywhere and what's available right now. The biggest problems are OS distributions that have older Python built-in, and this leads some tools to remain stuck in the past to support the OS distributions that are, similarly, stuck in the past.
As far as I can tell, Ubuntu 16.4 has Python 3.5 as the default. This is really great and it's a big help for those of us who want the better speed and the increased flexibility of Python 3. The next two chapters will look at the language changes and the migration strategies in detail. Before diving in it helps to have an overview of the alternatives available.
One choice is to convert the old applications and libraries, the idea is to discard Python 2 and not look back. I've given this the gold star of quality because it leads to great results. It's also possible to track two parallel versions of an application, in some cases it might be necessary to provide some support for an old version that's still in use. The old version can have bug fixes applied but all the new effort goes into the Python 3 version.
In the most complex cases, it may be necessary to support both versions, the six module is one way to do this, the 2to3 script can also be used. This is a decision that shouldn't be taken lightly. I'll provide some concrete guidance on when this is appropriate and when it may be too much work. Python 3 has been around for several years, there are no real risks, the ecosystem of libraries and packages has also matured.
What's left is for developers to finish the effort by converting the final layer of applications, tools and packages in use every day. This generally means all the enterprise applications and research tools that are still in Python 2. In the next section of this chapter I'll review the official migration guide, it's a seven-step process, but two of the steps aren't really needed anymore.
- Reviewing the differences between the two versions
- Reviewing the syntax changes introduced with Python 3
- Understanding the changes to built-in functions
- Reviewing the most important changes to the Python standard library
- Understanding which tools are required to migrate from Python 2.7 to Python 3
- Using six to handle class definitions
- Using six with standard library changes
- Using future
- Making syntax changes and class changes with futurize
- Using 2to3 or modernize