Functions, Modules, and I/O

Positional arguments are matched in order from the left, so a gets 1 and b gets 3. b has a default value of 2, but since a value was passed, that value takes priority. The unmatched 4 and 5 are gathered into args, marked with a single star, as a tuple, giving (4, 5). No keyword argument was passed at all, so kwargs is the empty dictionary {}, not None. If you think the default value 2 stays for b, you get (1, 2, (3, 4, 5), {}), but a positional argument takes priority over a default value. What goes into args is a tuple, not a list, so it is not displayed with square brackets either.

Only 1 is given as a positional argument, so a gets 1, and since b is not given, it stays at its default value of 2. Since there is no formal parameter named x in the definition, the keyword argument x=9 goes into kwargs, marked with a double star, as {'x': 9}. There is no leftover positional argument, so args is the empty tuple (). If you think x=9 goes into b, you get (1, 9, (), {}), but a keyword argument only goes into the formal parameter whose name matches. If you think the 9 goes into args, you get (1, 2, (9,), {}), but a value passed by name never falls back to the leftover positional arguments. Mixing up which star receives a dictionary and which receives a tuple could make it look as if {'x': 9} went into args and kwargs came out empty.

The default value [] is created just once, when the def line is executed, and that same list is stored on the function object. A new empty list is not created for each call, so the results of append carry over into the next call, and it grows as [1], [1, 2], [1, 2, 3]. If you think a new list is created each call, you get [1] / [2] / [3], but that is the behavior when lst=None is used as the default and a new one is built in the body instead. If you mistakenly write the body as return lst.append(x), None, the return value of append, is printed 3 times in a row. Also, if you write it not split across 3 lines but in one line as print(add(1), add(2), add(3)), all three refer to the same list, so [1, 2, 3], the state after everything has finished evaluating, is printed 3 times in a row.

A function object holds its default values in an attribute called __defaults__, as a tuple. Of the formal parameters, only lst has a default value, so it becomes a one-element tuple, displayed with a trailing comma. Its contents are the very list created just once at the def line, and since the results of appends from the 3 calls remain in it as is, it becomes ([1, 2, 3],). Before any call at all it would be ([],), but the effect of the calls showing up here is the frightening part of a mutable default argument. __defaults__ itself is a tuple, so it is not displayed with square brackets, nor does the list inside it turn into a tuple.

The expression for a default value is not evaluated every time the function is called; it is evaluated just once, at the moment the def line runs. At that time i is 5, so 5 is what gets stored on the function object. Even if i is reassigned to 10 afterward, the already-stored default value does not change, so h() returns 5. h(i), on the other hand, reads i at the moment of the call and passes it, so it returns the value at that time, 10. If you think the default value is evaluated at every call, both lines become 10, and if you think a value passed as an argument also replaces the default value, both lines become 5. The trap of a mutable default argument arises from this very same rule.

You cannot place a formal parameter without a default value after one that has a default value. This rule is checked at the syntax stage, so running the file first results in a SyntaxError, with the message non-default argument follows default argument. Since syntax parsing is performed on the whole file before execution, not even the print line runs. This is a completely different stage of detection from a TypeError caused by an error in how something is called. g(2, 3) itself matches the number of formal parameters, so if the definition were correct, 5 would be expected to display, but it never gets there. Writing it correctly means placing the one with a default value later, as in def g(b, a=1).

one finishes without ever passing through a return, so its return value is None. It is not an empty tuple or an empty string that is returned. two lines up 1 and 2 with a comma after return, but this is not returning two values; it is returning a single tuple, (1, 2). So type(two()).__name__ is tuple. If you think multiple return values are gathered into a list, you get list, but an expression lined up with commas creates a tuple. Stumbling when trying to assign and use the return value of a displaying function is often caused by this None.

When you put a single star in front of a list on the calling side, its contents are handed out one at a time as positional arguments. So a gets 1 and b gets 2, and the leftover 3 is gathered into args, marked with a single star, as a tuple, giving (3,). Since it is a one-element tuple, a trailing comma is attached. No keyword argument was passed, so kwargs is the empty dictionary {}, not None. If you forget the star and write f(vals), the list itself goes into the first formal parameter, giving ([1, 2, 3], 2, (), {}), which changes the result without raising an error. What goes into args is a tuple, so it is not displayed with square brackets either.

An annotation is merely recorded in the function object's __annotations__ dictionary; the type is not checked at runtime. Even though a is written as int, you can pass a string, and even though b is written as str, you can pass an integer. Even though the return value is written as bool, what is actually returned is the string it received, as is. So a string is printed as is on line 1. What is stored in __annotations__['a'] on line 2 is the type object int, so printing it displays it in the form <class 'int'>. It does not just print the 3 characters int. If you actually want to check types, you need to use a separate type-checking tool, or check it yourself in the body.

The standard pattern is to set the default value to None, and then check at the top of the body whether it is None and create a new empty list if so. Because None is a value that cannot be changed, no sharing problem arises even though it is evaluated only once. Since a new empty list is created each time for the first and second calls, which omit lst, they become [1] and [2], without any elements from the previous call remaining. On the third call, the caller passes [9], so lst is None becomes false, and 3 is appended to the end of the list that was passed, giving [9, 3]. If left as lst=[] before the rewrite, the elements from the previous call would remain when omitted on the second call, giving [1, 2]. If, ignoring the list the caller passed, you always create a new list, you get [1] / [2] / [3], and the benefit of this pattern — using it as given, when it is given — is lost. If, instead of appending at the end, you insert at the front with insert(0, x), the third call becomes [3, 9].

LEGB stands for Local, Enclosing, Global, Built-in, representing a search from the innermost outward. First it looks inside the currently executing function, then inside the function enclosing it, then the module top level, and finally built-ins such as len and print. The search ends the moment something is found, so if the same name is used on the inside, the outer name is hidden. Searching from the outside in would mean a local variable made inside a function loses to a module-level variable of the same name, which does not match how it actually behaves. The mistake of swapping the order of Global and Enclosing is also common, but the enclosing function is more inward than the module top level.

Assigning to a name inside a function creates that name as a new local variable, and the module-level variable of the same name is completely unaffected. print(x) inside the function reads the local x, so local is printed, but print(x) after the function returns reads the module-level x, so global is printed. If you think the assignment inside the function reaches outside, both lines become local, but that would require a global x declaration at the top of the function. Conversely, if you think the assignment inside the function is ignored and the outer value is read, both lines become global, but a local variable is indeed created once an assignment is made. This asymmetry — reading reaches outward, but writing does not — is the basic rule of Python's scoping.

Since inner inside outer has nonlocal n written, the assignment to n reaches n in the enclosing function outer. n returned by outer after calling inner is 2. Since inner inside outer2 has no such declaration, n = 2 just creates a new local variable for inner and ends there, so outer2's n stays 1. So the output is 2 1. If you think both reach, you get 2 2, and if you think neither reaches, you get 1 1, but what makes the difference is that single line of nonlocal. Note also that nonlocal cannot be used unless a variable of the same name already exists in an enclosing function, and it cannot point to a module-top-level variable either. When you want to rewrite a top-level variable, use global.

When Python compiles a function's body, it decides that any name that has an assignment anywhere in that function is a local variable, in its entirety. This determination is made for the function as a whole, and the order of the lines is irrelevant. In this bad function, there is an assignment z = 1 on a later line, so z is treated as a local variable of this function. Then, at the point of print(z), no value has been put into it yet, so it becomes an UnboundLocalError. The wording of this exception's message can vary by Python version, but the type name being UnboundLocalError does not change. The top-level z = 100 is not referenced. If you think, because it is on a line before the assignment, that the outer value is read, you would get 100, but that is not how Python behaves. Nor does an unassigned local variable become None.

f has global n written in it, so n = 20 rewrites the module-top-level n. f itself returns 20, and the top-level n also becomes 20. g has no such declaration, so n = 30 just creates a local variable in g and does not affect the top-level n. g returns its own local 30. The n read by the final print is the top-level one, and since f already rewrote it, it is 20. If you think even g's assignment reaches outside, you get 20 30 30, and if you think global has no effect, you get 20 30 10. A global declaration is an instruction saying "send an assignment to this name to the top level"; note that the declaration is not needed if you are only reading the value.

A star import ordinarily brings in every name that does not start with an underscore, but if the module side has a list of strings called __all__, it is narrowed down to only the names listed there. In this example, __all__ lists only pub, so neither other nor hello is brought in. other does not start with an underscore, but once __all__ exists, its list takes priority. Being a function does not give it special treatment either, so hello is likewise not brought in. __all__ only has an effect on a star import; an import that names a specific name, such as from mymod import hello, or the approach of import mymod followed by mymod.hello(), is not blocked.

When you run a file directly with python3, '__main__' is put into __name__. When it is imported from another file, the file name without its extension, 'mymod2', is put in instead. So when run directly, the 2 lines mymod2: __main__ and direct execution are printed, and when imported, only the 1 line mymod2: mymod2 is printed. if __name__ == '__main__': is the standard idiom that uses this difference to separate out "processing that should only run when this file is executed directly." Swapping the combination around would mean the code meant for verifying it works also runs when imported, which defeats the purpose of this idiom. If both cases gave the same value, this separation could not be made in the first place.

sys.argv is a list lining up the values passed from the command line, and the script name itself is placed at its head. In this example, it becomes a 3-element list, something like ['args.py', 'foo', 'bar'], so its len is 3. sys.argv[1:], cut from index 1 onward, becomes ['foo', 'bar'], with the script name removed. If you think the number of arguments passed becomes the length as is, you get 2, but that overlooks counting the script name at the head. Thinking sys.argv[1:] becomes ['args.py', 'foo'] happens if you get the starting position of the slice wrong. If you run it with no arguments passed at all, the length is 1, and sys.argv[1:] is an empty list.

A module that has been imported once is cached in a dictionary called sys.modules. The second import just binds the name to the same module object already registered there, so neither the file being read again nor the body being run again occurs. You can confirm this by putting a print at the module's top level and seeing that it displays only once even after importing twice. Even if you import it under a different alias, it points to the same object. So, if you rewrite a variable of some module, the rewritten value is also visible from any other place that imports that module. Importing the same name twice does not cause an error, and if you want to reload the body while developing, you need to explicitly use a feature of importlib.

sys.prefix points to where the currently running Python is installed, and sys.base_prefix points to where the Python it was derived from is located. Outside a virtual environment, both are the same value, but inside an environment created with python3 -m venv, sys.prefix comes to point to the created environment's directory, while sys.base_prefix continues to point to the original Python, so the two end up differing. The presence of this discrepancy is the clue for the determination. Saying that matching means being inside a virtual environment is the reverse condition. sys.base_prefix is always defined even outside a virtual environment, so it cannot be judged by whether it is defined. sys.prefix is also always defined. The specific path strings and the version of pip differ by environment, so this relationship is the only thing worth memorizing.

write returns, as its return value, the number of characters it wrote. "one\ntwo\nthree\n" has one at 3 characters, two at 3 characters, and three at 5 characters, totaling 11, plus 3 newlines attached, giving a total of 14 characters. Forgetting to count the newlines gives 11. If you think the number of lines written is returned, you get 3, but what write counts is strictly the number of characters. Choosing None on the assumption that nothing is returned is also a common mistake, easy to confuse with a method like append or sort that returns None, but write returns a count. Also, write does not add a newline automatically, so when you want to separate lines, you need to write it into the string yourself.

readlines returns every line as a list, but the newline character at the end of each element is left in as is. So it becomes ['one\n', 'two\n', 'three\n']. If you think the newlines are stripped out, you get ['one', 'two', 'three'], but to obtain that you need to apply rstrip to each element. What returns the whole thing as a single string, without splitting, is read; readlines returns a list. If you think it treats the whole file as one element, wrapped up without splitting on newlines, you get ['one\ntwo\nthree\n'], but readlines splits into a new element at every newline. When you write for line in f and take lines out one at a time, the line handed to you likewise still has the newline attached.

A file object remembers its current position. The first read reads all the way from the start to the end, so a holds 14 characters including the newline, and the position moves to the end. The second read tries to read from the end, so there is nothing left and it returns an empty string. So len(a) is 14, and b == '' is True. If you think the same content is returned the second time, b == '' would be False, but to read the same content again you need to move the position back to the start with f.seek(0). len(a) becoming 0 would happen if the file were empty, or if the position were already at the end. It is a situation easy to mistake for a read failure when nothing is displayed, but the cause is that the position is at the end.

Inside a with block, the file is still open, so closed is False. It is closed automatically upon exiting the block, so closed after that becomes True. So False is displayed, followed by True. The purpose of using with is to ensure this closing process runs even if you forget to write it, and it closes even if an exception occurs inside the block and you exit partway through. You can also open multiple files together in a single with statement by separating them with commas, and each of them is closed the same way. Trying to read from a file object after it has been closed results in ValueError, so you cannot read it again outside the with block.

Opening with 'w' truncates the file to length 0 at that point, regardless of whether existing content is present. Following through in order, the first 'w' writes AAA, and the following 'a' is an append, so it becomes AAABBB. But the moment it is opened a third time with 'w', AAABBB is erased, and C is written into it, so what finally remains is just C. If you think all 3 opens were with 'a', you get AAABBBC, and if you think the last write is ignored, you get AAABBB. Overwriting from the start without truncating is the behavior of 'r+', in which case only the leading character would be replaced, as in CAABBB. The truncation happens at the moment of opening, not at the moment of writing, so opening with 'w' and then writing nothing at all leaves the file empty. If you intend to append, use 'a'; if you intend to remake it, use 'w' — express your intent through the mode.

Calling read on a file object after it has been closed results in ValueError, with the message I/O operation on closed file. Since the object itself remembers that it was closed, it is not automatically reopened just because it is needed. Nor does it return an empty string in the sense of "there is nothing to read" — that is the behavior of read after it has already been read all the way to the end. Something like FileNotFoundError, for when a file cannot be found, is a relative of OSError, but the exception for operating on a closed file is separate, and it is ValueError. Using a with statement structurally avoids the situation of trying to read after exiting the block in the first place.

Omitting the second argument of open makes it read-only, 'r'. Calling write on a file object opened as read-only results in io.UnsupportedOperation, with the message not writable attached. Since this is syntactically valid code, you only notice the mode mistake once you actually run it. If you want to overwrite, use 'w'; if you want to append to the end, use 'a'; and if you want both reading and writing, reopen it with 'r+'. Conversely, calling read on a file opened with 'w' or 'a' results in the same io.UnsupportedOperation, with not readable this time. You can think of the mode as declaring, at the point of that file object, what is permitted with it.

Passing a number to read reads only that many characters. The first 5 characters are the 3 characters of one, the 1 newline character, and the t of two, giving 'one\nt'. The position advances by the amount read, so the tell right after returns 5. The following seek(0) moves the position back to the start, so the final tell is 0. If you think read does not move the position, you get tell returning 0 both times, and if you think seek has no effect, you get tell returning 5 both times. If you mix up read's argument as being a number of lines rather than a number of characters, intending to read up to line 5, you would mistakenly pick the whole file, 'one\ntwo\nthree\n', and 14. Using seek(0) lets you read from the start again on the same file object.

JSON only has arrays; there is no such type as a tuple. So at the point of dumps, the tuple is converted into an array, and the output is {"t": [1, 2]}. When it is read back, the array is restored as a list too, so the type of back['t'] is list; it does not go back to a tuple. The information that it was originally a tuple is nowhere retained, so the type changes over the round trip. If you want to keep it as a tuple, you need to apply tuple() yourself to the value after reading it back. Parentheses never appear in JSON output, and a tuple is not converted into a string either. For the same reason, using an integer as a dictionary key gets it converted into a string by dumps, and it does not go back to an integer when read back either.

A JSON object's keys must be strings, so calling dumps on a dictionary with an integer key converts the key into a string, giving {"1": "a"} as the output. This succeeds normally; no exception is raised here. Note, though, that when read back, the key stays as a string and does not go back to an integer. The set on the second line has no corresponding type in JSON, so trying to call dumps on it results in TypeError, with the message Object of type set is not JSON serializable. If you think a set gets written out as an array, you get [1, 2], but if you want that conversion, you need to apply list() to it yourself before passing it in. For values containing Japanese text, they are escaped in the output by default, and passing false to ensure_ascii outputs them as the literal characters.