Mastering Idiomatic Python Programming Techniques

 
I
d
i
o
m
a
t
i
c
 
P
y
t
h
o
n
 
J
a
m
e
s
 
E
d
w
a
r
d
s
 
W
h
a
t
 
a
r
e
 
i
d
i
o
m
s
?
 
Natural language: “expressions or phrases that
characterise a language”
 
Programming: clean and simple way of expressing
one particular (generally simple) task
 
W
h
y
 
s
h
o
u
l
d
 
I
 
u
s
e
 
t
h
e
m
?
 
1.
Readability
Idioms are often easy to read
 
2.
Maintainability
As it’s easier to understand it’s easier to modify in the
future
 
3.
Peer-review
Source code is becoming more frequently requested
by journals
 
C
a
v
e
a
t
 
Y
o
u
 
d
o
n
t
 
a
l
w
a
y
s
 
h
a
v
e
 
t
o
 
f
o
l
l
o
w
 
t
h
e
s
e
!
 
Learn by doing. Over time:
Do some programming
Read good source code and spot the patterns
Review your own code
Spot the flaws and identify where it can be improved
Repeat
 
S
i
m
p
l
e
r
 
e
x
a
m
p
l
e
s
 
S
w
a
p
p
i
n
g
 
V
a
l
u
e
s
 
Traditionally, use a temporary variable:
temp = a
a = b
b = temp
 
The Python idiom:
b, a = a, b
 
C
h
e
c
k
i
n
g
 
f
o
r
 
S
u
b
s
t
r
i
n
g
s
 
Make sure to use the “in” keyword:
if “world” in “Hello world”:
    print(“Found”)
 
But not for single characters:
if “-+” in “-+*/”:
    print(“This prints”)
# Use a tuple/set instead
if “-+” in (“-”, “+”, “*”, “/”):
    print(“Not found”)
 
U
s
i
n
g
 
d
i
c
t
i
o
n
a
r
i
e
s
 
You can access dictionary values like this:
veg_dict = {“pepper”: 1, “carrot”: 2}
key = “potato”
value = veg_dict[key] # raises a KeyError
 
You may get an exception. To protect against this,
you may want to use the get method:
value = veg_dict.get(key) # sets value to None
 
And if you want to supply your own default value:
value = veg_dict.get(key, default=“no vegetable”)
 
L
o
o
p
i
n
g
 
 
1
 
Using indexing in loops? This is the anti-idiom:
sentence = (“Hello”, “world!”)
for index in range(len(sentence)):
    print(sentence[index])
 
The Pythonic Way:
for word in sentence:
    print(word)
 
L
o
o
p
i
n
g
 
 
2
 
What about if you need the index? You could do this:
index = 0
for word in sentence:
    print(index, word)
    index += 1
 
But there’s a better way! Use the enumerate function:
for index, word in enumerate(sentence):
    print(index, word)
 
L
o
o
p
i
n
g
 
 
3
 
Reverse looping as encountered in many languages:
for index in range(len(sentence)-1, -1, -1):
    print(sentence[index])
 
Remember, Python has “batteries included”:
for word in reversed(sentence):
    print(word)
 
There are many similarly useful built in functions,
such as 
zip
, 
map
, 
sorted
, 
filter
 and others
 
L
o
o
p
i
n
g
 
 
4
 
No need to manually extract from a list of tuples/lists:
people = [(“Joe”, “Bloggs”), (“Josie”, “Bloggs”)]
for person in people:
    first_name, surname = person[0], person[1]
    print(first_name, surname)
 
You can extract tuple/list values natively:
for first_name, surname in people:
    print(first_name, surname)
 
L
o
o
p
i
n
g
 
 
5
 
(
y
e
s
,
 
m
o
r
e
)
 
How to loop over dictionary keys only:
words = {1: “hello”, 2: “world”}
for key in words:
    print(key)
 
Or if you want to delete or add key/value pairs to a
dictionary in a loop:
for key in list(words.keys()): # makes a copy
    if key == 2:
        del words[key]
 
L
o
o
p
i
n
g
 
 
6
 
Need dictionary values only? No need to do this:
for key in words:
    print(words[key])
 
Instead, use this method:
for word in words.values():
    print(word)
 
And finally, for both the key and value:
for key, word in words.items():
    print(key, word)
 
S
t
i
c
k
i
n
g
 
s
t
r
i
n
g
s
 
t
o
g
e
t
h
e
r
 
This is generally bad for lengthy lists (performance),
but is OK if it’s only a few:
sentence = “”
for word in words:
    sentence += “ ” + word
 
Instead, use join with a list:
sentence = “ ”.join(words)
 
NB: CPython 3+ is pretty fast with appending byte
strings, Unicode strings should still use join.
 
R
e
a
l
l
y
 
l
o
n
g
 
s
t
r
i
n
g
s
 
You could do something like this:
long_text = “This is a really, really, really,” +\
            “ really, really, really, long string”
 
But, Python lets you just use brackets:
long_text = (“
This is a really, really, really, ”
really, really, really long string”
)
 
C
h
e
c
k
i
n
g
 
f
o
r
 
i
d
e
n
t
i
t
y
 
Use the “is” keyword for identity:
a = “A”
letter = a
letter is a # evaluates to True
 
The difference between equality and identity:
other = “a”.upper()
other == a # evaluates to True
other is a # evaluates to False
 
C
h
e
c
k
i
n
g
 
f
o
r
 
i
d
e
n
t
i
t
y
 
The “
is not
 
construction is its own operator:
2 is not 3   # evaluates to True
 
You can see it behaves differently to “
is
” and “
not
separately:
2 is (not 3) # False, as “not 3” evaluates False
 
C
h
e
c
k
i
n
g
 
f
o
r
 
N
o
n
e
 
When checking for None, don’t test for equality:
if thing == None:
    print(“Works, but should be used sparingly”)
 
Instead, use an identity check (“
is
”):
if thing is None:
    print(“Preferred method to check for None”)
 
F
i
l
e
 
h
a
n
d
l
i
n
g
 
Clunky, and also risks leaving file handle open if
close() is forgotten:
file_handle = open(“file.txt”, “w”)
file_handle.write(“Hello world”)
file_handle.close()
 
Using the “with” keyword:
with open(“file.txt”, “w”) as file_handle:
    file_handle.write(“Hello world”)
 
U
s
e
 
t
h
e
 
s
t
a
n
d
a
r
d
 
l
i
b
r
a
r
y
 
Don’t reinvent the wheel. Standard library includes
lots of modules you may need:
Various file format parsers
Regular expressions & other text tools
Advanced logging system
Basic testing packages
Debugger and profiler
Data compression
High-level threading and multiprocessing
And lots of others!
 
U
s
e
 
t
h
e
 
P
y
P
I
 
PyPI
 is the Python Package Index – a vast number of
packages to do many things you might want to
 
Others have done the hard work so you don’t have to:
fuzzywuzzy: fuzzy string matching
scikit-learn: machine learning libraries
Flask, Django etc.: web frameworks
MPInfo: lookup information on UK Members of
Parliament
 
M
o
r
e
 
s
o
p
h
i
s
t
i
c
a
t
e
d
v
e
r
n
a
c
u
l
a
r
 
L
i
s
t
 
c
o
m
p
r
e
h
e
n
s
i
o
n
s
 
Constructing a simple list?
squares = list()
for i in range(10):
    squares.append(i**2)
 
Try this instead:
squares = list(i**2 for i in range(10))
 
You can also impose conditions:
sq3 = list(i**2 for i in range(10) if i % 2 == 0)
 
R
e
v
e
r
s
i
n
g
 
d
i
c
t
 
m
a
p
p
i
n
g
s
 
Use a dictionary comprehension:
boring_dict = {“first”: “A”, “second”: “B”}
exciting_dict = {val: key for key, val in
  
     boring_dict.items()}
 
Note, this particular construct:
only works for dictionaries with immutable values,
such as strings, integers, tuples etc.
doesn’t work with list values
 
A
 
r
u
l
e
 
o
f
 
t
h
u
m
b
 
f
o
r
c
o
m
p
r
e
h
e
n
s
i
o
n
s
 
I
f
 
t
h
e
 
c
o
m
p
r
e
h
e
n
s
i
o
n
 
i
s
 
l
o
n
g
e
r
 
t
h
a
n
 
a
 
l
i
n
e
,
 
c
o
n
s
i
d
e
r
u
s
i
n
g
 
a
 
n
o
r
m
a
l
 
f
o
r
 
l
o
o
p
!
 
Comprehensions are often less readable for more
complex expressions:
synth_signal = ((int(x[0]) | (int(x[1]) << 32))
  
    for x in np.array([alarm_word,
     
     mask_word]).T)
 
C
h
e
c
k
i
n
g
 
f
o
r
 
o
v
e
r
l
a
p
s
 
residents = [“Alice”, “Bob”, “Oskar”]
staff = [“Alex”, “Oskar”, “Divya”]
 
Naïve implementation of intersection:
intersection = list(person for person in residents
   
  if person in staff)
 
Better is to use sets (usually also faster):
intersection = set(residents) & set(staff)
 
M
u
l
t
i
p
l
e
 
c
o
m
p
a
r
i
s
o
n
s
 
Unnecessary:
if 1 < years and years < 5:
    print(“More than 1 but less than 5 years”)
 
Chained (transient) comparisons:
if 1 < years < 5:
    print(“More than 1 but less than 5 years”)
 
H
a
n
d
l
i
n
g
 
f
a
i
l
u
r
e
s
 
Look Before You Leap (LBYL):
if os.path.exists(“critical.dat”):
    # Another process may have deleted the file
    with open(“critical.dat”) as file_handle:
        print(file_handle.readlines())
else:
    print(“critical.dat has gone!”)
 
L
o
o
k
 
B
e
f
o
r
e
 
Y
o
u
 
L
e
a
p
 
Race conditions can appear (as just seen)
 
Intent may be obscured: lots of checks to verify
conditions before doing what you want
 
Anticipating all error conditions can get ugly
 
H
a
n
d
l
i
n
g
 
f
a
i
l
u
r
e
s
 
Easier to Ask Forgiveness than Permission (EAFP):
try:
    with open(“critical.dat”) as file_handle:
        print(file_handle.readlines())
except FileNotFoundError:
    print(“critical.dat has gone!”)
 
Ask yourself: is the code more readable with EAFP?
If so, then it’s Pythonic
But it has drawbacks…
 
E
a
s
i
e
r
 
t
o
 
A
s
k
 
f
o
r
F
o
r
g
i
v
e
n
e
s
s
 
Possible side effects if try block has too much code
Take care to only wrap the code that can raise
exceptions with the try statement
 
Can’t be used within comprehensions, generators or
lambda functions
 
T
r
y
/
E
x
c
e
p
t
/
E
l
s
e
/
F
i
n
a
l
l
y
 
Use else when the try block succeeds:
external_db = database.open_connection()
try:
    external_db.add_person(“James”, “Edwards”)
except ValidationError:
    print(“Database cannot validate new record”)
else: # no exception raised
    print(“Added record to database”)
finally: # 
always
 carried out
    external_db.close_connection()
 
L
o
o
p
i
n
g
 
 
7
 
Need to detect a particular break condition in a for
loop? For example, you could use a “sentinel”:
found = False # this is a sentinel
for i in range(7):
    if i == 8:
        found = True
        print(“Found 8”)
        break
if not found:
    print(“8 not found”)
 
L
o
o
p
i
n
g
 
 
7
 
However, for loops can use an “else” clause instead:
for i in range(7):
    if i == 8:
        print(“Found 8”)
        break
else:
    print(“8 not found”)
 
Else clause reached when the loop isn’t broken with a
break or return
 
S
u
b
t
l
e
t
i
e
s
 
o
f
 
s
e
n
t
i
n
e
l
s
 
This won’t work as you might expect:
def super_append(element, the_list=[]):
    the_list.append(element)
    return the_list
 
>>> super_append(1)
?
>>> super_append(2)
?
 
S
u
b
t
l
e
t
i
e
s
 
o
f
 
s
e
n
t
i
n
e
l
s
 
This won’t work as you might expect:
def super_append(element, the_list=[]):
    the_list.append(element)
    return the_list
 
>>> super_append(1)
[1]
>>> super_append(2)
[1, 2]
 
S
u
b
t
l
e
t
i
e
s
 
o
f
 
s
e
n
t
i
n
e
l
s
 
To get the correct behaviour, use a sentinel value and
modify the function to detect it:
def super_append(element, the_list=None):
    if the_list is None: # None most commonly used
        the_list = list()
    the_list.append(element)
    return the_list
>>> super_append(1)
[1]
>>> super_append(2)
[2]
 
T
h
a
n
k
s
 
f
o
r
 
l
i
s
t
e
n
i
n
g
 
E
x
t
r
a
 
r
e
s
o
u
r
c
e
s
 
Some resources/people with good advice:
PEP8
 – Official style guide for Python
Appropriate use of idioms
Hitchhikers Guide to Python: Code Style
Raymond Hettinger
 – excellent talks on code style
Jeff Knupp
 (but keep the zealotry in mind)
Python idioms
 (YouTube)
Google
Slide Note
Embed
Share

Idioms in Python, whether in natural language or programming, are expressions that simplify tasks and enhance code readability and maintainability. Using idioms in coding can also facilitate peer review. While not mandatory, learning idioms through practice, reading good source code, and self-review can lead to improvement over time. The content covers the benefits of idiomatic Python, exemplified by techniques like swapping values, checking substrings, using dictionaries efficiently, and looping the Pythonic way.


Uploaded on Jul 27, 2024 | 1 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Idiomatic Python Idiomatic Python James Edwards James Edwards

  2. What are idioms? What are idioms? Natural language: expressions or phrases that characterise a language Programming: clean and simple way of expressing one particular (generally simple) task

  3. Why should I use them? Why should I use them? 1. Readability Idioms are often easy to read 2. Maintainability As it s easier to understand it s easier to modify in the future 3. Peer-review Source code is becoming more frequently requested by journals

  4. Caveat Caveat You don t always have to follow these! Learn by doing. Over time: Do some programming Read good source code and spot the patterns Review your own code Spot the flaws and identify where it can be improved Repeat

  5. Simpler examples Simpler examples

  6. Swapping Values Swapping Values Traditionally, use a temporary variable: temp = a a = b b = temp The Python idiom: b, a = a, b

  7. Checking for Substrings Checking for Substrings Make sure to use the in keyword: if world in Hello world : print( Found ) But not for single characters: if -+ in -+*/ : print( This prints ) # Use a tuple/set instead if -+ in ( - , + , * , / ): print( Not found )

  8. Using dictionaries Using dictionaries You can access dictionary values like this: veg_dict = { pepper : 1, carrot : 2} key = potato value = veg_dict[key] # raises a KeyError You may get an exception. To protect against this, you may want to use the get method: value = veg_dict.get(key) # sets value to None And if you want to supply your own default value: value = veg_dict.get(key, default= no vegetable )

  9. Looping Looping 1 1 Using indexing in loops? This is the anti-idiom: sentence = ( Hello , world! ) for index in range(len(sentence)): print(sentence[index]) The Pythonic Way: for word in sentence: print(word)

  10. Looping Looping 2 2 What about if you need the index? You could do this: index = 0 for word in sentence: print(index, word) index += 1 But there s a better way! Use the enumerate function: for index, word in enumerate(sentence): print(index, word)

  11. Looping Looping 3 3 Reverse looping as encountered in many languages: for index in range(len(sentence)-1, -1, -1): print(sentence[index]) Remember, Python has batteries included : for word in reversed(sentence): print(word) There are many similarly useful built in functions, such as zip, map, sorted, filter and others

  12. Looping Looping 4 4 No need to manually extract from a list of tuples/lists: people = [( Joe , Bloggs ), ( Josie , Bloggs )] for person in people: first_name, surname = person[0], person[1] print(first_name, surname) You can extract tuple/list values natively: for first_name, surname in people: print(first_name, surname)

  13. Looping Looping 5 (yes, more) 5 (yes, more) How to loop over dictionary keys only: words = {1: hello , 2: world } for key in words: print(key) Or if you want to delete or add key/value pairs to a dictionary in a loop: for key in list(words.keys()): # makes a copy if key == 2: del words[key]

  14. Looping Looping 6 6 Need dictionary values only? No need to do this: for key in words: print(words[key]) Instead, use this method: for word in words.values(): print(word) And finally, for both the key and value: for key, word in words.items(): print(key, word)

  15. Sticking strings together Sticking strings together This is generally bad for lengthy lists (performance), but is OK if it s only a few: sentence = for word in words: sentence += + word Instead, use join with a list: sentence = .join(words) NB: CPython 3+ is pretty fast with appending byte strings, Unicode strings should still use join.

  16. Really long strings Really long strings You could do something like this: long_text = This is a really, really, really, +\ really, really, really, long string But, Python lets you just use brackets: long_text = ( This is a really, really, really, really, really, really long string )

  17. Checking for identity Checking for identity Use the is keyword for identity: a = A letter = a letter is a # evaluates to True The difference between equality and identity: other = a .upper() other == a # evaluates to True other is a # evaluates to False

  18. Checking for identity Checking for identity The is not construction is its own operator: 2 is not 3 # evaluates to True You can see it behaves differently to is and not separately: 2 is (not 3) # False, as not 3 evaluates False

  19. Checking for None Checking for None When checking for None, don t test for equality: if thing == None: print( Works, but should be used sparingly ) Instead, use an identity check ( is ): if thing is None: print( Preferred method to check for None )

  20. File handling File handling Clunky, and also risks leaving file handle open if close() is forgotten: file_handle = open( file.txt , w ) file_handle.write( Hello world ) file_handle.close() Using the with keyword: with open( file.txt , w ) as file_handle: file_handle.write( Hello world )

  21. Use the standard library Use the standard library Don t reinvent the wheel. Standard library includes lots of modules you may need: Various file format parsers Regular expressions & other text tools Advanced logging system Basic testing packages Debugger and profiler Data compression High-level threading and multiprocessing And lots of others!

  22. Use the Use the PyPI PyPI PyPI is the Python Package Index a vast number of packages to do many things you might want to Others have done the hard work so you don t have to: fuzzywuzzy: fuzzy string matching scikit-learn: machine learning libraries Flask, Django etc.: web frameworks MPInfo: lookup information on UK Members of Parliament

  23. More sophisticated More sophisticated vernacular vernacular

  24. List comprehensions List comprehensions Constructing a simple list? squares = list() for i in range(10): squares.append(i**2) Try this instead: squares = list(i**2 for i in range(10)) You can also impose conditions: sq3 = list(i**2 for i in range(10) if i % 2 == 0)

  25. Reversing Reversing dict dict mappings mappings Use a dictionary comprehension: boring_dict = { first : A , second : B } exciting_dict = {val: key for key, val in boring_dict.items()} Note, this particular construct: only works for dictionaries with immutable values, such as strings, integers, tuples etc. doesn t work with list values

  26. A rule of thumb for A rule of thumb for comprehensions comprehensions If the comprehension is longer than a line, consider using a normal for loop! Comprehensions are often less readable for more complex expressions: synth_signal = ((int(x[0]) | (int(x[1]) << 32)) for x in np.array([alarm_word, mask_word]).T)

  27. Checking for overlaps Checking for overlaps residents = [ Alice , Bob , Oskar ] staff = [ Alex , Oskar , Divya ] Na ve implementation of intersection: intersection = list(person for person in residents if person in staff) Better is to use sets (usually also faster): intersection = set(residents) & set(staff)

  28. Multiple comparisons Multiple comparisons Unnecessary: if 1 < years and years < 5: print( More than 1 but less than 5 years ) Chained (transient) comparisons: if 1 < years < 5: print( More than 1 but less than 5 years )

  29. Handling failures Handling failures Look Before You Leap (LBYL): if os.path.exists( critical.dat ): # Another process may have deleted the file with open( critical.dat ) as file_handle: print(file_handle.readlines()) else: print( critical.dat has gone! )

  30. Look Before You Leap Look Before You Leap Race conditions can appear (as just seen) Intent may be obscured: lots of checks to verify conditions before doing what you want Anticipating all error conditions can get ugly

  31. Handling failures Handling failures Easier to Ask Forgiveness than Permission (EAFP): try: with open( critical.dat ) as file_handle: print(file_handle.readlines()) except FileNotFoundError: print( critical.dat has gone! ) Ask yourself: is the code more readable with EAFP? If so, then it s Pythonic But it has drawbacks

  32. Easier to Ask for Easier to Ask for Forgiveness Forgiveness Possible side effects if try block has too much code Take care to only wrap the code that can raise exceptions with the try statement Can t be used within comprehensions, generators or lambda functions

  33. Try/Except/Else/Finally Try/Except/Else/Finally Use else when the try block succeeds: external_db = database.open_connection() try: external_db.add_person( James , Edwards ) except ValidationError: print( Database cannot validate new record ) else: # no exception raised print( Added record to database ) finally: # always always carried out external_db.close_connection()

  34. Looping Looping 7 7 Need to detect a particular break condition in a for loop? For example, you could use a sentinel : found = False # this is a sentinel for i in range(7): if i == 8: found = True print( Found 8 ) break if not found: print( 8 not found )

  35. Looping Looping 7 7 However, for loops can use an else clause instead: for i in range(7): if i == 8: print( Found 8 ) break else: print( 8 not found ) Else clause reached when the loop isn t broken with a break or return

  36. Subtleties of sentinels Subtleties of sentinels This won t work as you might expect: def super_append(element, the_list=[]): the_list.append(element) return the_list >>> super_append(1) ? >>> super_append(2) ?

  37. Subtleties of sentinels Subtleties of sentinels This won t work as you might expect: def super_append(element, the_list=[]): the_list.append(element) return the_list >>> super_append(1) [1] >>> super_append(2) [1, 2]

  38. Subtleties of sentinels Subtleties of sentinels To get the correct behaviour, use a sentinel value and modify the function to detect it: def super_append(element, the_list=None): if the_list is None: # None most commonly used the_list = list() the_list.append(element) return the_list >>> super_append(1) [1] >>> super_append(2) [2]

  39. Thanks for listening Thanks for listening

  40. Extra resources Extra resources Some resources/people with good advice: PEP8 Official style guide for Python Appropriate use of idioms Hitchhikers Guide to Python: Code Style Raymond Hettinger excellent talks on code style Jeff Knupp (but keep the zealotry in mind) Python idioms (YouTube) Google

More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#