Understanding Python Essentials

 
Hands-on Python Tutorial
 
Martin Styner
 
What is Python?
 
Python is Interpreted
 − Python is processed at runtime by the
interpreter. You do not need to compile your program before
executing it.
Python is Interactive
 − You can actually sit at a Python prompt and
interact with the interpreter directly to write your programs.
Python is a Beginner's Language
 − Python is a great language for the
beginner-level programmer.
Python is Object-Oriented
 − Python supports Object-Oriented style
or technique of programming that encapsulates code within objects.
 
Python vs Shell scripting
 
Python is better at complex tasks
Shell scripts are better at file handling
 
Caution: Differences to shell scripts
Python cares about “white space” (tabs, space)
Indents have meaning as they group the code
There is no “endif” or ”end” that serves as a delimiter to group the code
Multi-line statements can be done with “\” (not needed for arrays)
 
Python setup
 
Via Anaconda package (download for your local computer), use
python v3
Use one of the Python installations in /NIRAL/tools/Python
/NIRAL/tools/Python/Python-3.8.0 should work on janus or most servers
These are for general python stuff, not deep learning
Edit your .auxcshrc to
setenv PATH ${PATH}:/tools/Python/Python-3.8.0/bin
 
Running Python scripts
 
As notebooks in jupyter (more later), in interactive fashion, great for
developing scripts and plots etc
Running: python3 <myscript>.py
Adding the python3 binary as the first line of the script:
#!/NIRAL/tools/Python/Python-3.8.0
Now you can run it as a simple executable
 
Editing Python scripts
 
Use your favorite editor, emacs or gedit or vim
They all have python specific modes that help with editing
You can keep using your favorite editor
Best: use Sublime “subl mypython.py” as it has the best python
specific mode
Or do it in Jupyter (next)
 
Starting “jupyter notebook”
 
Locally: simply type ”jupyter notebook” in terminal -> starts up into your
local browser (best first go to the folder where your Docs are)
Over ssh tunneling (a bit more complicated):
Log into janus: ssh 
onyen@janus.ia.unc.edu
”cd” to your folder with the python scripts:
cd /NIRAL/tools/Python/Examples/MergingCSV/
Start jupyter and make it ready for ssh: jupyter notebook --no-browser --port 8888
This will yield the connection information for your browser, e.g.
http://localhost:8888/?token=585695ca6164f82fb56d0a9feb57d7694298d94917df4a25
In another shell, create an ssh tunnel: ssh -NL 8888:localhost:8888
onyen@janus.ia.unc.edu
Start jupyter window in your local browser using the localhost link
 
Merging CSV’s in python with pandas
 
Open “CombineCSVs.ipynb”
.ipynb is a python notebook. You need to convert it to a script if you want to run it
outside jupyter
 You can see the output of the last run when loading
A notebook is composed of cells that can be run independently
Run Cells => run the currently selected cells
<Shift> <Return> runs the current cell
First cell => import the packages you need
import <package>
import <package> as <nickname>
from <package> import <part-of-the-package>
 
Pandas Package
 
Documentation on pandas.pydata.org
Uses dataframes to store data, lots of operations can be applied to those
dataframes
Reading CSV: read_csv(filepath)
Setting the index/ID column via set_index
Assumption: header – rows with column keys
To combine CSV, there has to be a column that identifies the row-entry
such as subject ID
 
First look at the data
 
Just type the name of the variable
 
 
Key Index
 
Selecting Data / Counting Data
 
Use “[ ]” to select parts of the data
data[‘key’]  => only the key column
Data[row criterior] => only the rows satisfying
this criterion
Use count() to count non-zero element
subDataV06.count()
Use Counter function to count rows
 
Merging Dataframes
 
Use “merge”
Can merge two dataframes at a time
Several important parameters
validate => check whether data/key is unique with ‘one_to_one’
left_index = True, right_index=True => both dataframes should have the keys
and they should both be used in the merging
how => merge style
‘inner’ = needs key to be present in both dataframes (this is what you usually want)
‘outer’ = needs key to be present at least in one (but not necessary in both) dataframes
sort => also sort the rows using the key column
 
Selecting Multiple Columns from Dataframes
 
How can I remove many columns at once?
Use the loc function and a column selector
Use “~” as a “NOT” operator to remove certain columns
columns.isin() select multiple columns
“:” is whole row (or column)
 
Saving to CSV
 
Use “to_csv” dataframe function
 
Exporting python notebook to script
 
File -> Download as -> Python
Best clean it up after the export =>   CombineCSVs_edit.py
Run it
python3  CombineCSVs_edit.py
Slide Note
Embed
Share

Python is an interpreted language that is beginner-friendly and supports object-oriented programming. It excels at complex tasks and interactive script development. The setup process, running scripts, and editing can be done efficiently, and Jupyter notebooks provide an interactive environment for script development. Learn how to set up Python via Anaconda, run scripts in Jupyter, and use various editors effectively.


Uploaded on Aug 03, 2024 | 2 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Hands-on Python Tutorial Martin Styner

  2. What is Python? Python is Interpreted Python is processed at runtime by the interpreter. You do not need to compile your program before executing it. Python is Interactive You can actually sit at a Python prompt and interact with the interpreter directly to write your programs. Python is a Beginner's Language Python is a great language for the beginner-level programmer. Python is Object-Oriented Python supports Object-Oriented style or technique of programming that encapsulates code within objects.

  3. Python vs Shell scripting Python is better at complex tasks Shell scripts are better at file handling Caution: Differences to shell scripts Python cares about white space (tabs, space) Indents have meaning as they group the code There is no endif or end that serves as a delimiter to group the code Multi-line statements can be done with \ (not needed for arrays)

  4. Python setup Via Anaconda package (download for your local computer), use python v3 Use one of the Python installations in /NIRAL/tools/Python /NIRAL/tools/Python/Python-3.8.0 should work on janus or most servers These are for general python stuff, not deep learning Edit your .auxcshrc to setenv PATH ${PATH}:/tools/Python/Python-3.8.0/bin

  5. Running Python scripts As notebooks in jupyter (more later), in interactive fashion, great for developing scripts and plots etc Running: python3 <myscript>.py Adding the python3 binary as the first line of the script: #!/NIRAL/tools/Python/Python-3.8.0 Now you can run it as a simple executable

  6. Editing Python scripts Use your favorite editor, emacs or gedit or vim They all have python specific modes that help with editing You can keep using your favorite editor Best: use Sublime subl mypython.py as it has the best python specific mode Or do it in Jupyter (next)

  7. Starting jupyter notebook Locally: simply type jupyter notebook in terminal -> starts up into your local browser (best first go to the folder where your Docs are) Over ssh tunneling (a bit more complicated): Log into janus: ssh onyen@janus.ia.unc.edu cd to your folder with the python scripts: cd /NIRAL/tools/Python/Examples/MergingCSV/ Start jupyter and make it ready for ssh: jupyter notebook --no-browser --port 8888 This will yield the connection information for your browser, e.g. http://localhost:8888/?token=585695ca6164f82fb56d0a9feb57d7694298d94917df4a25 In another shell, create an ssh tunnel: ssh -NL 8888:localhost:8888 onyen@janus.ia.unc.edu Start jupyter window in your local browser using the localhost link

  8. Merging CSVs in python with pandas Open CombineCSVs.ipynb .ipynb is a python notebook. You need to convert it to a script if you want to run it outside jupyter You can see the output of the last run when loading A notebook is composed of cells that can be run independently Run Cells => run the currently selected cells <Shift> <Return> runs the current cell First cell => import the packages you need import <package> import <package> as <nickname> from <package> import <part-of-the-package>

  9. Pandas Package Documentation on pandas.pydata.org Uses dataframes to store data, lots of operations can be applied to those dataframes Reading CSV: read_csv(filepath) Setting the index/ID column via set_index Assumption: header rows with column keys To combine CSV, there has to be a column that identifies the row-entry such as subject ID

  10. First look at the data Just type the name of the variable Key Index

  11. Selecting Data / Counting Data Use [ ] to select parts of the data data[ key ] => only the key column Data[row criterior] => only the rows satisfying this criterion Use count() to count non-zero element subDataV06.count() Use Counter function to count rows

  12. Merging Dataframes Use merge Can merge two dataframes at a time Several important parameters validate => check whether data/key is unique with one_to_one left_index = True, right_index=True => both dataframes should have the keys and they should both be used in the merging how => merge style inner = needs key to be present in both dataframes (this is what you usually want) outer = needs key to be present at least in one (but not necessary in both) dataframes sort => also sort the rows using the key column

  13. Selecting Multiple Columns from Dataframes How can I remove many columns at once? Use the loc function and a column selector Use ~ as a NOT operator to remove certain columns columns.isin() select multiple columns : is whole row (or column)

  14. Saving to CSV Use to_csv dataframe function

  15. Exporting python notebook to script File -> Download as -> Python Best clean it up after the export => CombineCSVs_edit.py Run it python3 CombineCSVs_edit.py

More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#