Welcome to LWN.netThe following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider accepting the trial offer on the right. Thank you for visiting LWN.net! |
|
The Python 3.8 beta cycle is already underway, with Python 3.8.0b1 released on June 4, followed by the second beta on July 4. That means that Python 3.8 is feature complete at this point, which makes it a good time to see what will be part of it when the final release is made. That is currently scheduled for October, so users don't have that long to wait to start using those new features.
The walrus operator
The headline feature for Python 3.8 is also its most contentious. The process for deciding on PEP 572 ("Assignment Expressions") was a rather bumpy ride that eventually resulted in a new governance model for the language. That model meant that a new steering council would replace longtime benevolent dictator for life (BDFL) Guido van Rossum for decision-making, after Van Rossum stepped down in part due to the "PEP 572 mess".
Out of that came a new operator, however, that is often called the "walrus operator" due to its visual appearance. Using ":=" in an if or while statement allows assigning a value to a variable while testing it. It is intended to simplify things like multiple-pattern matches and the so-called loop and a half, so:
m = re.match(p1, line)
if m:
return m.group(1)
else:
m = re.match(p2, line)
if m:
return m.group(2)
else:
m = re.match(p3, line)
...
if m := re.match(p1, line):
return m.group(1)
elif m := re.match(p2, line):
return m.group(2)
elif m := re.match(p3, line):
...
ent = obj.next_entry()
while ent:
... # process ent
ent = obj.next_entry()
while ent := obj.next_entry():
... # process ent
Debug support for f-strings
The f-strings (or formatted strings) added into Python 3.6 are quite useful, but Pythonistas often found that they were using them the same way in debugging output. So Eric V. Smith proposed some additional syntax for f-strings to help with debugging output. The original idea came from Larry Hastings and the syntax has gone through some changes, as documented in two feature-request issues at bugs.python.org. The end result is that instead of the somewhat cumbersome:
print(f'foo={foo} bar={bar}')
print(f'{foo=} {bar=}')
>>> foo = 42
>>> bar = 'answer ...'
>>> print(f'{foo=} {bar=}')
foo=42 bar=answer ...
Beyond that, some modifiers can be used to change the output, "!s" uses the str() representation, rather than the default repr() value and "!f" will be available to access formatting controls. They can be used as follows:
>>> import datetime
>>> now = datetime.datetime.now()
>>> print(f'{now=} {now=!s}')
now=datetime.datetime(2019, 7, 16, 16, 58, 0, 680222) now=2019-07-16 16:58:00.680222
>>> import math
>>> print(f'{math.pi=!f:.2f}')
math.pi=3.14
One more useful feature, though it is mostly cosmetic (as is the whole feature in some sense), is the preservation of spaces in the f-string "expression":
>>> a = 37
>>> print(f'{a = }, {a = }')
a = 37, a = 37
Positional-only parameters
Another change for 3.8 affords pure-Python functions the same options for parameters that those implemented in C already have. PEP 570 ("Python Positional-Only Parameters") introduces new syntax that can be used in function definitions to denote positional-only arguments—parameters that cannot be passed as keyword arguments. For example, the builtin pow() function must be called with bare arguments:
>>> pow(2, 3)
8
>>> pow(x=2, y=3)
...
TypeError: pow() takes no keyword arguments
But if pow() were a pure-Python function, as an alternative Python implementation might want, there is no easy way to force that behavior. A function could accept only *args and **kwargs, then enforce the condition that kwargs is empty, but that obscures what the function is trying to do. There are other reasons described in the PEP, but many, perhaps most, are not things that the majority of Python programmers will encounter very often.
Those that do, however, will probably be pleased that they can write a pure-Python pow() function, which will behave the same as the builtin, as follows:
def pow(x, y, z=None, /):
r = x**y
if z is not None:
r %= z
return r
def fun(a, b, /, c, d, *, e, f):
...
fun(1, 2, 3, 4, e=5, f=6) # legal
fun(1, 2, 3, d=4, e=5, f=6) # legal
fun(a=1, b=2, c=3, d=4, e=5, f=6) # illegal
A movable __pycache__
The __pycache__ directory is created by the Python 3 interpreter (starting with 3.2) to hold .pyc files. Those files contain the byte code that is cached after the interpreter compiles .py files. Earlier Python versions simply dropped the .pyc file next to its .py counterpart, but PEP 3147 ("PYC Repository Directories") changed that.
The intent was to support multiple installed versions of Python, along with the possibility that some of those might not be CPython at all (e.g. PyPy). So, for example, standard library files could be compiled and cached by each Python version as needed. Each would write a file of the form "name.interp-version.pyc" into __pycache__. So, for example, on my Fedora system, foo.py will be compiled when it is first used and __pycache__/foo.cpython-37.pyc will be created.
That's great from an efficiency standpoint, but may not be optimal for other reasons. Carl Meyer filed a feature request asking for an environment variable to tell Python where to find (and put) these cache files. He was running into problems with permissions in his system and was disabling cache files as a result. So, he added a PYTHONPYCACHEPREFIX environment variable (also accessible via the -X pycache_prefix=PATH command-line flag) to point the interpreter elsewhere for storing those files.
And more
Python 3.8 will add a faster calling convention for C extensions based on the existing "fastcall" convention that is used internally by CPython. It is exposed in experimental fashion (i.e. names prefixed with underscores) for Python 3.8, but is expected to be finalized and fully released in 3.9. The configuration handling in the interpreter has also been cleaned up so that the language can be more easily embedded into other programs without having environment variables and other configuration mechanisms interfere with the installed system Python.
There are new features in various standard library modules as well. For example, the ast module for processing Python abstract syntax trees has new features, as do statistics and typing. And on and on. The draft "What's New In Python 3.8" document has lots more information on these changes and many others. It is an excellent reference for what's coming in a few months.
The status of PEP 594 ("Removing dead batteries from the standard library") is not entirely clear, at least to me. The idea of removing old Python standard library modules has been in the works for a while, the PEP was proposed in May, and it was extensively discussed after that. Removing unloved standard library modules is not particularly controversial—at least in the abstract—until your favorite module is targeted, anyway.
The steering council has not made a pronouncement on the PEP, nor has it delegated to a so-called BDFL-delegate. But the PEP is clear that even if it were accepted, the changes for 3.8 would be minimal. Some of the modules may start raising the PendingDeprecationWarning exception (many already do since they have been deemed deprecated for some time), but the main change will be in the documentation. All of the 30 or so modules will be documented as being on their way out, but the actual removal will not happen until the 3.10 release—three or so years from now.
The future Python release cadence is still under discussion; currently Python 3.9 is scheduled for a June 2020 release, much sooner than usual. Python is on an 18-month cycle, but that is proposed to change to nine months (or perhaps a year). In any case, we can be sure that Python 3.8 will be here with the features above (and plenty more) sometime before Halloween on October 31.
