Mastering Idiomatic Python Programming Techniques
Idioms in Python, whether in natural language or programming, are expressions that simplify tasks and enhance code readability and maintainability. Using idioms in coding can also facilitate peer review. While not mandatory, learning idioms through practice, reading good source code, and self-review can lead to improvement over time. The content covers the benefits of idiomatic Python, exemplified by techniques like swapping values, checking substrings, using dictionaries efficiently, and looping the Pythonic way.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Idiomatic Python Idiomatic Python James Edwards James Edwards
What are idioms? What are idioms? Natural language: expressions or phrases that characterise a language Programming: clean and simple way of expressing one particular (generally simple) task
Why should I use them? Why should I use them? 1. Readability Idioms are often easy to read 2. Maintainability As it s easier to understand it s easier to modify in the future 3. Peer-review Source code is becoming more frequently requested by journals
Caveat Caveat You don t always have to follow these! Learn by doing. Over time: Do some programming Read good source code and spot the patterns Review your own code Spot the flaws and identify where it can be improved Repeat
Simpler examples Simpler examples
Swapping Values Swapping Values Traditionally, use a temporary variable: temp = a a = b b = temp The Python idiom: b, a = a, b
Checking for Substrings Checking for Substrings Make sure to use the in keyword: if world in Hello world : print( Found ) But not for single characters: if -+ in -+*/ : print( This prints ) # Use a tuple/set instead if -+ in ( - , + , * , / ): print( Not found )
Using dictionaries Using dictionaries You can access dictionary values like this: veg_dict = { pepper : 1, carrot : 2} key = potato value = veg_dict[key] # raises a KeyError You may get an exception. To protect against this, you may want to use the get method: value = veg_dict.get(key) # sets value to None And if you want to supply your own default value: value = veg_dict.get(key, default= no vegetable )
Looping Looping 1 1 Using indexing in loops? This is the anti-idiom: sentence = ( Hello , world! ) for index in range(len(sentence)): print(sentence[index]) The Pythonic Way: for word in sentence: print(word)
Looping Looping 2 2 What about if you need the index? You could do this: index = 0 for word in sentence: print(index, word) index += 1 But there s a better way! Use the enumerate function: for index, word in enumerate(sentence): print(index, word)
Looping Looping 3 3 Reverse looping as encountered in many languages: for index in range(len(sentence)-1, -1, -1): print(sentence[index]) Remember, Python has batteries included : for word in reversed(sentence): print(word) There are many similarly useful built in functions, such as zip, map, sorted, filter and others
Looping Looping 4 4 No need to manually extract from a list of tuples/lists: people = [( Joe , Bloggs ), ( Josie , Bloggs )] for person in people: first_name, surname = person[0], person[1] print(first_name, surname) You can extract tuple/list values natively: for first_name, surname in people: print(first_name, surname)
Looping Looping 5 (yes, more) 5 (yes, more) How to loop over dictionary keys only: words = {1: hello , 2: world } for key in words: print(key) Or if you want to delete or add key/value pairs to a dictionary in a loop: for key in list(words.keys()): # makes a copy if key == 2: del words[key]
Looping Looping 6 6 Need dictionary values only? No need to do this: for key in words: print(words[key]) Instead, use this method: for word in words.values(): print(word) And finally, for both the key and value: for key, word in words.items(): print(key, word)
Sticking strings together Sticking strings together This is generally bad for lengthy lists (performance), but is OK if it s only a few: sentence = for word in words: sentence += + word Instead, use join with a list: sentence = .join(words) NB: CPython 3+ is pretty fast with appending byte strings, Unicode strings should still use join.
Really long strings Really long strings You could do something like this: long_text = This is a really, really, really, +\ really, really, really, long string But, Python lets you just use brackets: long_text = ( This is a really, really, really, really, really, really long string )
Checking for identity Checking for identity Use the is keyword for identity: a = A letter = a letter is a # evaluates to True The difference between equality and identity: other = a .upper() other == a # evaluates to True other is a # evaluates to False
Checking for identity Checking for identity The is not construction is its own operator: 2 is not 3 # evaluates to True You can see it behaves differently to is and not separately: 2 is (not 3) # False, as not 3 evaluates False
Checking for None Checking for None When checking for None, don t test for equality: if thing == None: print( Works, but should be used sparingly ) Instead, use an identity check ( is ): if thing is None: print( Preferred method to check for None )
File handling File handling Clunky, and also risks leaving file handle open if close() is forgotten: file_handle = open( file.txt , w ) file_handle.write( Hello world ) file_handle.close() Using the with keyword: with open( file.txt , w ) as file_handle: file_handle.write( Hello world )
Use the standard library Use the standard library Don t reinvent the wheel. Standard library includes lots of modules you may need: Various file format parsers Regular expressions & other text tools Advanced logging system Basic testing packages Debugger and profiler Data compression High-level threading and multiprocessing And lots of others!
Use the Use the PyPI PyPI PyPI is the Python Package Index a vast number of packages to do many things you might want to Others have done the hard work so you don t have to: fuzzywuzzy: fuzzy string matching scikit-learn: machine learning libraries Flask, Django etc.: web frameworks MPInfo: lookup information on UK Members of Parliament
More sophisticated More sophisticated vernacular vernacular
List comprehensions List comprehensions Constructing a simple list? squares = list() for i in range(10): squares.append(i**2) Try this instead: squares = list(i**2 for i in range(10)) You can also impose conditions: sq3 = list(i**2 for i in range(10) if i % 2 == 0)
Reversing Reversing dict dict mappings mappings Use a dictionary comprehension: boring_dict = { first : A , second : B } exciting_dict = {val: key for key, val in boring_dict.items()} Note, this particular construct: only works for dictionaries with immutable values, such as strings, integers, tuples etc. doesn t work with list values
A rule of thumb for A rule of thumb for comprehensions comprehensions If the comprehension is longer than a line, consider using a normal for loop! Comprehensions are often less readable for more complex expressions: synth_signal = ((int(x[0]) | (int(x[1]) << 32)) for x in np.array([alarm_word, mask_word]).T)
Checking for overlaps Checking for overlaps residents = [ Alice , Bob , Oskar ] staff = [ Alex , Oskar , Divya ] Na ve implementation of intersection: intersection = list(person for person in residents if person in staff) Better is to use sets (usually also faster): intersection = set(residents) & set(staff)
Multiple comparisons Multiple comparisons Unnecessary: if 1 < years and years < 5: print( More than 1 but less than 5 years ) Chained (transient) comparisons: if 1 < years < 5: print( More than 1 but less than 5 years )
Handling failures Handling failures Look Before You Leap (LBYL): if os.path.exists( critical.dat ): # Another process may have deleted the file with open( critical.dat ) as file_handle: print(file_handle.readlines()) else: print( critical.dat has gone! )
Look Before You Leap Look Before You Leap Race conditions can appear (as just seen) Intent may be obscured: lots of checks to verify conditions before doing what you want Anticipating all error conditions can get ugly
Handling failures Handling failures Easier to Ask Forgiveness than Permission (EAFP): try: with open( critical.dat ) as file_handle: print(file_handle.readlines()) except FileNotFoundError: print( critical.dat has gone! ) Ask yourself: is the code more readable with EAFP? If so, then it s Pythonic But it has drawbacks
Easier to Ask for Easier to Ask for Forgiveness Forgiveness Possible side effects if try block has too much code Take care to only wrap the code that can raise exceptions with the try statement Can t be used within comprehensions, generators or lambda functions
Try/Except/Else/Finally Try/Except/Else/Finally Use else when the try block succeeds: external_db = database.open_connection() try: external_db.add_person( James , Edwards ) except ValidationError: print( Database cannot validate new record ) else: # no exception raised print( Added record to database ) finally: # always always carried out external_db.close_connection()
Looping Looping 7 7 Need to detect a particular break condition in a for loop? For example, you could use a sentinel : found = False # this is a sentinel for i in range(7): if i == 8: found = True print( Found 8 ) break if not found: print( 8 not found )
Looping Looping 7 7 However, for loops can use an else clause instead: for i in range(7): if i == 8: print( Found 8 ) break else: print( 8 not found ) Else clause reached when the loop isn t broken with a break or return
Subtleties of sentinels Subtleties of sentinels This won t work as you might expect: def super_append(element, the_list=[]): the_list.append(element) return the_list >>> super_append(1) ? >>> super_append(2) ?
Subtleties of sentinels Subtleties of sentinels This won t work as you might expect: def super_append(element, the_list=[]): the_list.append(element) return the_list >>> super_append(1) [1] >>> super_append(2) [1, 2]
Subtleties of sentinels Subtleties of sentinels To get the correct behaviour, use a sentinel value and modify the function to detect it: def super_append(element, the_list=None): if the_list is None: # None most commonly used the_list = list() the_list.append(element) return the_list >>> super_append(1) [1] >>> super_append(2) [2]
Thanks for listening Thanks for listening
Extra resources Extra resources Some resources/people with good advice: PEP8 Official style guide for Python Appropriate use of idioms Hitchhikers Guide to Python: Code Style Raymond Hettinger excellent talks on code style Jeff Knupp (but keep the zealotry in mind) Python idioms (YouTube) Google
