Comprehensive Overview of IEEE Python Seminar by Tim York at SIUE ECE Department
Explore the detailed insights shared by Tim York during the IEEE Python Seminar held at SIUE ECE Department on Dec 7, 2016. He covered topics such as ways to get Python on different operating systems, basics of Python, numeric analysis, string parsing, Python on SBCs, reasons to choose Python, getting started with Python on various platforms, recommended IDEs, and more.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
IEEE Python Seminar TIM YORK SIUE ECE DEPARTMENT DEC 7 2016
Outline Best way to get Python: Windows (WinPython Distribution) MacOS (MacPorts / Homebrew) Linux (Use your package manager (ie. apt-get, dnf, emerge, ) The basics (Types, loops, functions) Numeric Analysis (Python can work great as a MATLAB replacement I can show you how to plot said data) String Parsing (I'll show you how to get data from a text file) Python on SBCs (ie. Raspberry Pi / BeagleBone) GPIO library, I2C, SPI
Why Python? It works pretty much everywhere! (Linux, MacOS, Windows, Intel, ARM...) It's pretty high level --> Tons of External Libraries to do just about anything! It's also quick to prototype an idea It can be scripted, interpreted, some mixture of both It plays well with C Many APIs are straight C, a lot of them also have Python APIs Can be used to speed up slow Python code String processing is great! Web interaction is great! Numeric Analysis is great! Machine Learning is great! GUIs are pretty easy!
To Begin How Do I Get Python? For Windows, there are standard distributions (www.python.org) However, dependencies are annoying, PIP sometimes works I much prefer WinPython (https://winpython.github.io) It's standalone (you don't need Admin to run it, although you can make it your Windows version of Python with 1-click as admin) It's updated semi-regularly If a package doesn't exist, look here: (http://www.lfd.uci.edu/~gohlke/pythonlibs/) For MacOS, it comes with Python! PIP to install libraries I usually use MacPorts (www.macports.org) -- Huge collection of Open Source software for Mac, almost like a Linux PM Keeps your main Python and dependencies up to date Linux It's in your package manager (and probably already there by default): Ubuntu/Debian/Mint -- sudo apt-get python-... Fedora/CentOS -- sudo dnf install python-...
IDEs I'm going to use Spyder for this seminar, as it is most like MATLAB Spyder has a nice editor, interactive window, auto-complete, auto- format, etc. There are plenty others...use whichever you like (IDLE comes with Python) Python doesn't use braces, semicolons, or anything, it uses SPACES! 4 SPACES Marks the scope of a function, loop, class, etc. NOT TABS!!! Your IDE should automatically do this for you I will upload my Spyder history from the seminar afterwords
The Basics Python, at its core, is an interpreted language, meaning you will be writing scripts Ostensibly, there is no main, however, you can make one if necessary Python just runs scripts one line after the other NOTE: Since very line must be interpreted, it can be slow, just like MATLAB (although many times you won't notice!) Python has no declared types, the interpreter tries to figure out the type in the context of what you are doing: 2+3 returns 5 (as 2 and 3 are integers, so is 5) '2' + '3' returns '23' (the ' ' makes it a char, "A" + "B" also returns "AB", single and double quotes are essentially interchangeable!) A + B returns an error (A and B aren't defined as numbers or strings) All numbers are integers unless defined with a . (ie 12345 is an integer, 12345. is a floating point double) 3 / 4 returns 0, but 3.0 / 4.0 returns 0.75 (3.0 / 4 also returns 0.75, any numeric operation with floating- point returns floating-point) Comments (Single line start with #, multi-line start with """ [3 double-quotes] and end with """)
Flow Control Standard flow control if condition: do stuff if condition evaluates to something not zero elif condition1: do stuff if condition1 evaluates to something not zero else: do stuff if previous conditions don't evaluate for ix in range(10): #This loops from 0 to 9 print ix for ix in range(5,15): #This loops from 5 to 14 for ix in range(15,5,-1): #This loops from 15 to 6 for ix in range(5,20,4): #This loops from 5 to 19 in steps of 4 (so 5, 9, 13, 17) while (test): do stuff while test evaluates to something not zero
Data Types Strings: ordered sequences of characters str1 = "Hello World!" str1[0] = 'H', str1[4] = 'o', str1[1:4] = 'ello' Lists: ordered sequences of any type list1 = [5,8,2,6,7] list2 = ["hello", "world", "this", "is", "a" , "sentence","."] list2[0] = 'Hello', list2[2] = "this", list2[0][0] = 'H' Set: unordered group of unique entries Dictionary: group of name: value pairs dict1 = {'username': 'tyork', 'password': '12345'} dict1['username'] returns 'tyork' The len() function returns the number of elements in a data type len(list1) = 5, len(str1) = 12, len(dict1) = 2
Data Types, cont. Many data types have specific methods built-in for them Say I have list1 = [5,8,2,6,7] but I need to add a 9 to the list. How do I do it? list1.append(9) makes list1 return [5,8,2,6,7,9] What if I don't need the 2 in there anymore? del list1[2] makes list1 return [5,8,6,7,9] Now I have a string str1 = "Hello World!". Strings have built-ins like .find(), .split(), .capitalize(), etc. More on this in a bit ... How to build a list in a loop (very common): builtList = [] for ix in range(10): builtList.append(ix) Basically, start with an empty list, append new elements to it
Functions/Libraries Functions can be declared like: def function1(arg1, arg2, ..): OR def function1(arg1=default1, arg2=default2, ) for optional args If you want to make a library, just put a bunch of functions in a .py file (Easiest) and put it in same directory as your main script or PATH Then import nameOfPyFile at top of code allows access to all functions like nameOfPyFile.function1(argIn1, argIn2) OR from nameOfPyFile import * accesses them like function1(argIn1, argIn2) OR, to import just function1 from nameOfPyFile import function1 OR, to import with a shorter namespace, import nameOfPyFile as nopf to access function1 as nopf.function1(argIn1, argIn2)
Cool, Python Can Work Just Like MATLAB!!! There are 3 core libraries that allow Python to work just like MATLAB: numpy (array and numeric operations), matplotlib (plotting functions and other MATLAB equivalents), and scipy (many other MATLAB equivalent functions) Spyder, as an IDE, will automatically import these and looks like MATLAB If not, start your code with: import numpy as np import matplotlib.pyplot as plt import scipy as sp NOTE: You don't have to use Spyder to use these libraries, Spyder is just an IDE! https://docs.scipy.org/doc/numpy-dev/user/numpy-for-matlab- users.html
Numpy arrays Numpy, like MATLAB, wants to do everything as an array Arrays are easy to create, especially from lists, just use the numpy asarray() function (ie. list1Array = np.asarray(list1)) Numpy funtions create arrays too: list1Array = np.arange(0,10) creates an array of numbers from 0 to 9 Numpy arrays ARE typed, though. Check types by using the .dtype built-in: list1Array.dtype returns int64 You can easily convert array types, just use the .astype built-in list1Array.astype(np.float) makes it floating point (OR list1AsFloat = list1Array.astype(np.float), list1AsFloat.dtype returns float64) Many of the numpy functions have a dtype= argument to specify type at time of creation (ie. array1 = np.arange(0,10,dtype=np.float) returns an array of 0 to 9, but as floats this time!)
Numpy array indexing Numpy arrays are indexed using [] instead of (), indexing starts at 0 (inclusive), and ends at a defined range (non-inclusive), : is the range operator, like MATLAB array1[0] = 0.0, array1[0:3] = array([0.0, 1.0, 2.0]) Arrays can be negatively indexed, starting from the end: array1[-1] = 9, array1[-4] = 6 You can specify the step size (start:stop:step): array1[0:8:2] = array([0.,2.,4.,6.]), array1[7:3:-1] = array([7.,6.,5.,4.]) 2D (and N-D arrays are a thing): array2 = np.random.rand(10,10) creates a 10 x 10 array of random numbers array2[5][4] = 0.1482, array2[5][2:5] = array([0.5262, 0.6219, 0.1482]) array3 = np.random.rand(20,10,30,5) creates a 20 x 10 x 30 x 5 array of random numbers array3[1][2][3][4]
Do Things As Array by Array (NOT FOR LOOPS!) Just like MATLAB, if I had two 100 x 100 arrays, if I want to multiply them element-by-element I would do: arr3 = arr1 * arr2 DO NOT DO: for ix in range(100): for jx in range(100): arr3[ix][jx] = arr1[ix]*arr2[jx] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^--- THIS IS HORRIBLY INEFFICIENT! Also, most numpy functions are designed to operate on whole arrays at a time Why? Most numpy functions are light wrappers over C code. For example, the arr1 * arr2 is really just a python wrapper around a C function called multiply that has: for (ix = 0; ix < 100; ix++){ for (jx = 0; jx < 100; jx++) { out[ix][jx] = in1[ix] * in2[jx]; } } There is a little bit more overhead to do the Python wrapping, but the bulk of the computation is done in compiled and optimized C, meaning it's pretty fast Double (or triple or N sized) for loops in Python require Python to interpret the for each time it iterates, plus the operation will be the Python operation So, a numpy multiply will call the interpreter once, the double for in Python calls the interpreter M*N times
Cool, Now How About Plotting? Matplotlib will plot things very nicely Simple Example, Plot A sine wave: import numpy as np import matplotlib.pyplot as plt t = np.arange(0,1,1e-3) #Make an array of values from 0 to 0.999 in steps of 0.001 y = 2.0*np.sin(2*np.pi*5*t) #Create a 5 Hz sine wave plt.plot(t,y) #Plot the waveform If not in Spyder,, plt.plot(t,y) must be followed by plt.show()
More Plotting Stuff: xlabel(), ylabel() title() legend() Save as PNG (default), SVG (Vector Graphics High Quality!), JPG, BMP, etc.
Like I Said, Strings Are Cool! That's all well and good for data created within numpy, what about data elsewhere? Python has great string processing capabilities! str1 = "This is a string to test for string processing!" As shown before, you can .find() --> Returns the position of the matched string (e.g. str1.find("test") returns 20 .split() --> Breaks string up into a list of subwords, based on what is passed to split (e.g str1.split() returns ['This', 'is', 'a', 'string', 'to', 'test', 'for', 'string', 'processing!']) .replace(arg1,arg2) --> Replace all instances of arg1 with arg2 (e.g. str1.replace("string", "parrot") returns "This is a parrot to test for parrot processing!"
Slightly More Complex Example: Grab a website, parse out data, plot: http://146.163.133.81:8080/D -- Contains timed temperature data from 381 lab We want to get the web page Get all time/value pairs Filter out all temperatures that are not between "0.0" and "130.0" Plot the time/temperature data
Grab The Website The requests library is basically a browser in Python: import requests #import the requests library s = requests.session() #Creates a session object to hold cookies, etc. webSite = s.get("http://146.163.133.81:8080/D") #Fetches the page webSiteLines = webSite.text.split('\n') #Split the page by lines
Now, Parse Out The Data We have the raw HTML data, but we really want the text. What pattern should we find? For the temperature data: How about "D: ", as that is what all temperatures start with. For the time data: All timestamps start with "2016" and end with "." (We won't worry about milliseconds) We will now iterate over all lines BUT, we should parse out the temperature value first in case it doesn't meet our "0" to "100" requirement (We won't need the timestamp if that's the case)
Parsing Code timeStamp = [] tempData = [] for ix in range(len(webPageLines)): if (webPageLines[ix].find("D:") > 0): stIdx = webPageLines[ix].find("D:") + 3 spIdx = webPageLines[ix].find("&", stIdx) tempVal = float(webPageLines[ix][stIdx:spIdx]) if (tempVal >= 0.0 and tempVal <= 130.0): tempData.append(tempVal)
Rats!!! There are some invalid temperatures! We are screwed, right? NOPE. Python can do exception handling: for ix in range(len(webPageLines)): if (webPageLines[ix].find("D:") > 0): stIdx = webPageLines[ix].find("D:") + 3 spIdx = webPageLines[ix].find("&", stIdx) try: tempVal = float(webPageLines[ix][stIdx:spIdx]) if (tempVal >= 0.0 and tempVal <= 130.0): TempData.append(tempVal) except: continue
Awesome, Now Last Thing, Let's Get the Timestamp if (tempVal >= 0.0 and tempVal <= 130.0): tempData.append(tempVal) stIdx = webPageLines[ix].find("2016") spIdx = webPageLines[ix].find(".",stIdx) tsVal = webPageLines[ix][stIdx:spIdx] timeStamp.append(tsVal)
Now, How To Plot? The time series is newest first. We would like oldest first. 2 Ways: 1: Use the .reverse() built-in for lists. This automagically reverses the order of timeStamp (and tempData)! 2: timeStamp[-1] is the last one in the list. So, from indexing, timeStamp[- 1::-1] would return the list in reverse order [start = -1 starts at last element, 1st : range operator, 2nd : means go to end, -1 do in reverse order Pros/Cons: Method1 alters the actual list, Method2 doesn't Great, we now have it in the proper order, but what do we do know? Plot the time series! But wait, matplotlib can't index a time series! Dang, foiled again! The Pandas library is designed to deal with time series (I told you there was a library for everything!)
Finally, The Plot! timeStamp.reverse() tempData.reverse() import pandas ts = pandas.Series(index=timeStamp, data=tempData) ts.plot()
That Was Painful. Let's Write It To A File File I/O is pretty easy too! f = open('temperatureData.txt', 'w') for ix in range(len(timeStamp)): f.write('{0}\t{1}\n'.format(timeStamp[ix], tempData[ix])) f.close() open has two arguments: 1st is the filename (a string), 2nd is the mode (also a string) --> Mode is same as C ('r' read, 'w' write, 'a' append, 'rb' read binary...) ' '.format() is sort of like printf. {0} gets replaced by the 1st argument, {1} by the second, etc.
Now, To Complete The Circle, Read The File Back in and Plot f = open('temperatureData.txt', 'r') tempDataFile = f.readlines() f.close() timeStampIn = [] tempDataIn = [] for ix in range(len(tempDataFile)): lineSplitOnTab = tempDataFile[ix].split('\t') timeStampIn.append(lineSplitOnTab[0]) tempDataIn.append(float(lineSplitOnTab[1][0:-1])) Necessary to get rid of \n!
How Will I Ever Use This? Ever try to program a Raspberry Pi or BeagleBone? Python makes it easy (Special Thanks to Adafruit!) GPIO Libraries I2C Libraries (Really just light wrappers for SMBus, though) SPI Libraries (I'll show you an NRF24 Exmaple) ADC Library (For BeagleBone) All of these can be great data sources (ie. An analog sensor, an I2C accelrometer, etc.) Use your Python knowledge to parse/format the data/analyze/plot, etc.
Other Advanced Stuff Open CV For Computer Vision Polarization Image Processing (I'll show you my scripts) Read RAW images from Raspberry Pi camera (Custom Library)
