It’s quite rare for mature programming languages to introduce new operators. I suppose there’s no good reason for this; if a new operator is useful and doesn’t break anything existing in the language, then it’s a win. But it feels like it’s much easier to persuade people that a language needs a new standard library function than new syntax.
As Python 3.8 gradually spreads into common usage, now might be a good time to look at one of the big changes it introduced, assignment expressions or the affectionately-nicknamed “walrus operator”:
while (block := f.read(256)) != '':
process(block)
That := is the new kid on the block.
What you can do with this
The most common reason this will be useful is where you want to use an if, while or other conditional operation and you also want to assign to a variable while you’re doing it.
If you do much text processing with regular expressions, you’ve probably come across this a lot:
m = re.match(r'(\d+)', input)
if m:
do_stuff(m.group(1))
If you’ve come to Python from another language, you may at first be surprised that you can’t write this in a more compact way:
if m = re.match(r'(\d+)', input):
do_stuff(m.group(1))
This looks pretty tempting, but it isn’t legal syntax. But the new operator allows you to do this:
if m := re.match(r'(\d+)', input):
do_stuff(m.group(1))
This fixes a minor annoyance, but it doesn’t seem like a big deal. However, in the real world programmers tend to prefer writing compact code even if it’s less efficient, and will write things like:
if len(options) > max_options:
print(f"Too many options: {len(options)}")
This calls len() twice, just to print the error message. In a case like this it isn’t going to matter, but in performance-critical code where the duplicated operation was something more expensive this could turn out to be significant.
Why is this even necessary?
The obvious question is: Why doesn’t Python let you write the expression you want to write with a simple = operator?
Other programming languages manage this just fine. C++ does this all the time:
while (iter++ != collection.end()) {
//...
}
You can do it in Ruby:
while line = gets # process line end
You can do it in Java:
while (n = input.nextInt()) {
System.out.println("You entered " + n);
}
You can do it in Javascript:
while (x = x - 1) {
console.log(`x is ${x}`);
}
Is Python just being deliberately difficult here?
A digression about expressions
Programming languages vary immensely, but in general you can distinguish statements and expressions.
An expression is a chunk of code that results in some value, such as 1 + 2 or name.reverse().
Expressions are powerful because you can (typically) use an expression anywhere a value is expected, including inside another expression. Therefore you can have arbitrarily complicated expressions.
A statement is a chunk of code that results in some action or state change, such as import left_pad or print("hello " + name) or num_socks = 2 * num_feet.
The body of a function (or a module or other code block) is a series of statements.
The obvious question is whether there’s any overlap between expressions and statements. An expression on its own can be treated as a trivial statement, which just evaluates the value and discards it. For example, in Python you can write:
def my_func():
1 +1
The line 1 + 1 doesn’t do much here: the value is calculated and then discarded, and the function returns None. More usefully, some expressions will have side-effects:
def speak_my_weight(person):
audio.speak(person.weight)
Here the audio.speak(person.weight) is an expression (it calls a function and yields a value) that’s being used as a statement, because of its side effects.
So an expression is a statement, but is a statement also an expression? That depends which language you’re using. There are three possibilities:
- Statements are never able to be expressions (unless they are trivial)
- Every statement is an expression
- It depends on the statement
The first option isn’t really desirable. Languages like Lisp and Ruby go for the second option. Python takes the third.
Why would you want all statements to be expressions?
The nice thing about the rule “every statement yields a value” is that it’s really easy to explain, and really easy to remember.
Languages become more expressive by having a small number of rules that can be combined in lots of different ways; that way you get maximum power while taxing the programmer’s brain the minimum amount.
So in Ruby, for example, you can do something like:
songType = if song.mp3Type == MP3::Jazz
if song.written < Date.new(1935, 1, 1)
Song::TradJazz
else
Song::Jazz
end
else
Song::Other
end
The fact that an if block returns a value means that you don’t have to do an explicit assignment in the branches of the block. The assignment is done only once. This is a little bit forced in this case, but you can imagine if the destination of the assignment was something complex it might be nice not to have to repeat it.
Why wouldn’t you want all statements to be expressions?
Having a small number of rules that can be combined in infinite ways is very elegant, but there are always edge cases where the human brain doesn’t work like that, which can lead to confusion.
Every C programmer has done this once in their life:
if (a = 10) {
printf("a is 10\n");
}
This fails because it’s assigning to a, not checking its value. The reason this compiles at all is that C takes the view that an assignment is an expression that yields a value, so there’s no reason you shouldn’t be able to use the resultant value in an if condition. The fact that this is plainly not what a human being would want is no concern of the C compiler.
“But wait!”, I hear you cry, “my linter would have caught that mistake. There’s no need to forbid it in the language spec when tooling can catch it.”
This is a fair position, but the truth is messier. If you have such a linter rule enabled on every project you work on and every team you work with, then it might as well be fixed in the language. If you don’t, you’ll get confused when you switch teams.
If you break the rule frequently, then you’ll have to have ugly annotations to disable the linter all over the place. If you break the rule very rarely, why is it such a big deal if the language forces you to work around it in a few rare cases?
The Python way
Python generally chooses explicit but slightly more verbose code over simpler code that can trip people up. You can judge that it makes the wrong decision if you like, but the language can’t please everyone.
Therefore Python doesn’t allow assignment expressions to yield a value that can be used in an expression.
Hang on a minute…
If you’re paying attention, you may have been starting to wonder about the Python construction:
a = b = c = 10
This technique for initialising multiple variables is popular and available in a lot of languages (though personally I’ve never found it useful). It’s often possible for a language to compile this by treating it this way:
a = (b = (c = 10))
This wouldn’t make any sense in Python, because c = 10 isn’t an expression so can’t be assigned to b. What’s going on here?
Python simply treats this as a special case. An assignment in Python can have multiple targets, and so Python chooses to treat the expression a = b = c = 10 as one single assignment, with a value of 10 and 3 targets: a, b and c.
I think this is an example of Python taking the practical path: it seems somehow neater and simpler if you don’t have to have a special rule to deal with multiple assignments, but it doesn’t actually save much complexity in the Python implementation. In return for biting the bullet and treating this as a special case, developers who write in Python benefit from a language that more frequently does what they expect.