2022 / Learning Python
Learning Python
2022
Introduction
In this paper I have recorded my accounts, lessons, and reflections on learning how to program with Python through an application-based project.
I am a third-year undergraduate student studying chemistry, and I want to focus my work on atmospheric chemistry in my graduate & post-graduate and/or industry research. I love what I learn in this field and every day I see dots of seemingly chaotic and unconnectable complexities get connected. Each new figure showing concentration vs time, oxidation products of terpenes, radiance vs wavenumber, and so on gives more meaning to what’s going on around me. And so I’ve asked myself “How can I do more? How can I be a better chemist? How can I accelerate this learning process so that, sooner rather than later, I too can contribute to further understanding?” This whole paper is a recollection of one of the more important answers I’ve identified to that question: learn Python.
Fortunately, I’ve been exposed to some atmospheric research, specifically that by Dr. Ezra Wood and his group attempting to improve the sensitivity of their peroxy radical sensor. The biggest takeaway from this experience was seeing first-hand how essential computational analysis is. Being able to analyze and organize the hundreds and thousands of data points per day and then evaluate it for all types of trends is the brunt of the research. This process is extensive but critical. No chemist can succeed without strong skills in these areas. And I want to succeed — because, well, I want to contribute to further understanding.
I got to the answer of my aforementioned, driving question by speaking with professors, fellow students, and other researchers... I learned that the most robust and transferrable skill needed to perform these types of data organization, model construction, and computational analyses is to learn how to program — ideally with a language that excels with such tasks. As I said, I chose Python.
Here's how I learned it.
*I didn’t fully learn it. I don’t think I ever will. I just liked how a 5-word sentence sounded to close out the introduction. I feel I’ve ruined that effect now though.
Ideation
Reaching Out
I knew way before I started this process that I wanted to learn Python by completing a project. I figured it would be an exciting way to learn something new as I’d have complementary motivations all at play. I can learn the programming, learn the project, and have a tangible outcome to reflect on. My first problem, then, was to conceive a proper and achievable project idea. I wasn’t sure of the extent to which one project would be more apt to teaching programming basics vs another — let alone the capabilities of programming on its own. Because the ideal subject of said project was something that combined Python and atmospheric chemistry, I asked my (soon to be at the time) atmospheric chemistry professor if he had any ideas/suggestions.
He did! Dr. Wood put me in contact with Dr. Shannon Capps who leads the Atmospheric Modeling group at Drexel (the existence of which was unbeknownst to me at the time). I had never heard of her work or spoke with her before, but she kindly responded to my cold email with enthusiasm and an idea that eventually became the topic of my project. Dr. Capps and I met over zoom not long thereafter, and the plan was set in motion.
The Project Task
The Atmospheric Modeling group is attempting to propagate sensitivities from an outdoor, regional model they have to an indoor model that is written in Python. To do this, they need a specific package to mathematically utilize hyper-dual numbers. This hyper-dual library is currently written in Fortran, so it must be translated to Python. That’s my project.
When Dr. Capps and I first met, one of the first things I told her was that my knowledge of programming and computer science (I didn’t even know what terms to use) was very minimal. I wanted to be clear and up-front that though my ambition would let me take on any project idea she may be able to come up with (this hyper-dual package included), I should probably stick to something manageable and realistic. She appreciated this honesty, because ultimately this project is helping her group too, but did say that it is a realistic goal given the limited background I had. We both agreed that once I could demonstrate a very basic command of Python to Dr. Capps, I’d be given access to the Fortran hyper-dual library and could start working. I was excited to have found a project and ready to start.
One of the perks of this project is that translating code from one language to the next involves relatively little programmatic thinking — the focus is more on common syntax. The programming is already done, I just need to translate it to the proper language. That said, by being exposed to said programming, I can slowly pick it up as I go along. I need to know what exactly I am translating — later this proves to be an important aspect to the project.
Activation Energy
Figuring Out How to Talk to Computers
After the short turnaround between voicing my interest to Dr. Wood and then the conception of a full project plan, I was enthusiastic about embarking. I learned quickly however that learning to program is full of repeated self-starting and restarting. More than anything, it’s filled with a lot of links. Here’s some words I had journaled at the start:
- Step 1… figure out how to enter a Python operating framework…
- I just downloaded the Python installation package
- Learning what IDEs are and Code Editors… essential.
- https://realpython.com/python-ides-code-editors-guide/#what-are-ides-and-code-editors
- Step 2
- I’m realizing I don’t want to do too much without having a system set up to track my progress and build toward an article that follows my journey.
- I want to use NoodleTools (I used that in HS) but Drexel doesn’t have it. I chatted with Kathleen Turner at Drexel Libraries, and she guided me to use EndNote. I’m going to put an hour or so into setting this up on my computer and having it ready to start my ‘quest.’
- Note… something incredibly funny about this process is that there are absolutely so many links! I press one link that I think will get me to my answer, but there ends up being 43 more! I’m just trying to get the software I’m going to use to help me write the article… I still don’t know how to add 2 numbers in Python.
I never actually used EndNote, but I would say going down that rabbit hole and the many others at that stage were necessary to get started. It was the barrier I needed to overcome, and though a more direct and guided approach would’ve been more efficient, I do believe I learned more this way. Even though I didn’t use a lot of the information I read about, I learned how other people have done similar projects. I invested my time, cleared up my vision a bit, and humbled myself.
Basic Command of Python
I downloaded the Anaconda environment and started with the Jupyter Lab web-based IDE from the Anaconda Navigator. Anaconda, as I learned, is a Python and R programming distribution created for Data Science. The easiest way for beginners to think about it is a big box that contains everything you need to get started. With it comes Python, hundreds of Python libraries (like NumPy, Pandas, SciPy, etc.), and different IDEs to code in. An IDE is an Integrated Development Environment, meaning it is a place where one can write code, execute code, and perform other tasks related to the development of a program. An IDE is different than a regular Code Editor (like TextEdit, Notepad, and Notepad ++) because it allows you to do much more than just write and edit. I quickly found Jupyter Lab to be the best possible IDE to start with because your programs can be executed on the same document as your work. The interface is user-friendly and not overwhelming.
With my understanding of these platforms where it needed to be and the platforms themselves downloaded and installed, I was ready to write and execute my first Python code. Finally!
But how...
Eager and impatient, after finishing a movie on a Friday night, I displayed my MacBook screen on my living room TV and picked a random thumbnail that comes up on YouTube when you search “Python beginner tutorial.” I had Jupyter Lab open on my Mac and “Learn Python in 1 Hour” open on my TV at 12:30 AM Saturday morning... this is how!
I started coding and didn’t stop till I made it to the end. After all of the pauses, replays, and own attempts of following along, it wasn’t until 4:00 AM that I finished. The 1-hour tutorial taught me a lot of the basics very fast, and I was truly surprised at how quickly this language can be picked up. I found it extremely helpful to think of what I was doing in terms of things I already knew. Below is a few of the things I made note of early on that night:
Things I've Learned
-
Variables are the same things as the ones in Maple. Or like the ones on the TI-84 calculator. On the calculator when you hated repeatedly typing “$6.67430 \times 10^{-11}$” for the gravitational constant, you’d type it in and then hit the “store” button and store it as “G.” In Python, variables store something in the memory of the computer. Later on, you can use this something in a calculation or an action.
-
Using information is easy in Python; however, you need to make sure that your “expressions” or, for me (because I think of programming as a maple worksheet) “equations” use things that match up. For example, if you define the variable “happiness_level” to equal input(“Enter your happiness level: “), then the variable will be equal to something that is within quotes. Anything within quotes is called a string. A string is not a number and therefore you cannot perform the operation: 10 - “happiness_level”. Instead, you must make the type of values in the operations consistent.
-
Making value types consistent is done with 4 simple functions. First is the int() function which has the ability to seek within a string (something within quotations) and make it an integer number. Next is string() which makes a value a string value (opposite integer). Another useful function is float() which converts values to a floating point number (a number with a decimal). Last is the bool() function which converts a value to a Boolean (binary, true or false).
-
Getting information is important for learning Python, it’s a very useful way to grow your programming. The two most important commands (I think that’s the word?) for this are input(“ “) and print(). The input() command lets you make a prompt followed by a field that can be used to enter information. This information can then be manipulated by your program. Once manipulated, you can opt to display the results of what the code has done. Displaying these results is achieved by the print() command.
I enjoyed the teaching style of the video creator because he would talk through everything, he typed in a way that I could follow along and code with him. I would learn in the moment and have my own, slightly adapted, code to look back on after the fact.
Here are two of the earliest Python codes of mine:
In the first of the two codes above, I told the computer to let me give the program some information. When executed two input boxes appeared. One let me input the first value, and the next asked for the second. The program then gave me the sum. In the second code, I told the computer to print the answer to the logical statement “Python” in “Python for atmospheric chemist.”
There are a whole lot more of such codes in my (no better name than...) “Hello World” Python file.
After graduating from this initial tutorial, I enrolled in a proper online course offered by the same creator of the original video, Mosh Hamdani. I began learning more in-depth about these basic principles and finding new ways to build upon the most basic of concepts.
Here are some more of the early codes from this time:
- Here I am experimenting with lists. Lists are mutable objects that (usually) contain multiple things. What is meant by mutable is that you can change some of the things that are contained within them and manipulate said things.
- The first thing I do to the list that I named “numbers” was I inserted “-1” in the first index (position) and printed it.
- Next, I took out the index “3”.
- After this I asked the computer if “1” was within my list… The computer said yes by providing the Boolean “True “. I did the same thing again by asking if “10” was in my list and it said no with “False.” I also asked what the size is, or count, of the list, and it told me it was 5.
- Using the same list from last time, I am doing something new.
- First is a “for loop.” A for loop lets you repeat something a fixed number of times — this quantity is equal to the number of indexes of the object you are “repeating.” In Python (computer programming in general) this repeating action is called iterating.
- Iterating can be done over different types of objects like lists (what’s used here), ranges, tuples (comma separated lists within parentheses), sets, strings, and more.
- In the first case, I am calling “each_value” the variable for which the loop will iterate over each value of the list called “numbers.” The result is shown: “1” then “2” then “3” then “4” ... so on. Usually, other variable names are used instead of “each_value.” This just helped me understand.
- Next, I show the “while loop.” A while loop repeats something so long as some given condition is true. Here, the condition is that “i” (which I’ve set equal to 0) must be less than the length of a range of 0 to 5 (in Python the length of a range from 0 to 5 equals to 4. You start at and include 0 but end before the last number).
- Given true circumstances, the while loop will then return to me 0 because it is the first index of the range. Next, however, because I’ve specified i=i+1, the loop will iterate again and return 1 plus the last iteration. This loop continues until the circumstances are no longer true or the range ends. Whichever comes first.
Though (seemingly) simple, the loops discussed above carry with them important principles in Python programming. Having command of loops and understanding iteration is critical to high-level computations in this language. Python is an object-oriented program language, which means its programs heavily rely on the objects themselves. A Python programmer will find that the skill of iterating over objects is a common and essential task; therefore, making and manipulating loops is something I try to practice in different ways.
- Here is an extremely simple code that I am using to highlight a valuable tool: ternary operators. When learning about ternary operators, I quickly found that they will prove too valuable to complex code — even though they seem superfluous in simple ones like this. Here’s what a ternary operator is:
- In the above code, I’ve laid out a simple conditional statement: if age is less than or equal to 18, then say “eligible.” Whatever else age may be, say “ineligible.”
- What’s missing in my little translation is “message.” Message in this context acts as a ternary operator. What I’ve really done with this conditional statement, is defined “message” based on the condition of “age.”
- As you could probably see even at this early point, when handling a very large program, it could be much easier to have your program redefine variables on its own so, as the programmer, you don’t have to do it over and over again.
As you can see, my grasp of the language grew. However, I knew I’d need to start splitting my path away from straight web development (which is what Mosh focuses on) to more computational applications of the language. To do this, I decided I’d try my hand at plotting some data. This was an exciting point for me because I knew I had the rudimentary skills down, and I was able to create my own programs without the direct help of Mosh. That said, a learning curve awaited me. This one however was not like the rest. It was easier this time and I found this encouraging.
I knew that the first step was to accustom myself a bit with what packages are in Python. Aside from knowing about packages from this project itself, I had already read about them here and there when link surfing early in the process. From what I knew, packages are big programs that you can import into your own program to perform tasks that the Python language “default” can’t do (without of course making your program as equally complex as the package). If I wanted to write a relatively small code that would extract data and plot it, I figured I’d need to import a package.
I was right.
Thanks to a quick google search, I found that the main plotting package on Python is matplotlib. I imported it and another package NumPy like the person on the forum suggested. Then I started step 2... I tried something!
Below is my first plot. $f(x) = sin(x)$:
- Here I’ve defined “x” to be an evenly spaced interval of 30 numbers between 0 and 10.
- I defined “y” to be the sin(“x”).
- Last, I plotted “x” vs. “y” with black colored dots. Voila!
The next task I set for myself was to plot some data from excel. Another serviceable google search showed that to do this, I need to import the Pandas library to extract my data from excel.
Here’s how it went:
- The first step here was defining “data” to be the information in the excel sheet.
- Next, I defined my independent and dependent variables “Temperature” and “pressure” as the data shown in the corresponding columns of the excel sheet.
- An annoying issue I ran into is that when looking for data in an excel sheet, you can’t have your column names be anywhere other than row “1.” If the names are anywhere else, the “list(data[])” command won’t find it. I was plagued by this problem for too long than I’d like to admit. From now on, all my excel sheets will begin on row 1.
- After this of acquiring data, all that was really left was plot the relationship! Matplotlib formatted my axes’ intervals and smoothed out my data points by itself. I added a few formatting touches like axes labels and chart title, but that’s it.
I was having a lot of fun, but I knew that simple plotting wasn’t really challenging me. I wanted to try something a bit more sophisticated, so I decided I would try something that would let me plot, fit data, perform matrix multiplication, and integrate all in one.
I had some data of heat capacity at constant pressure of some sample collected at different temperatures and wanted to find the change in entropy of this sample from 100 K to 300K. The change in entropy can be expressed as:
$$
\Delta S = \int \frac{C_p}{T} \; \textit{d}T.
$$
I needed to find a functional form of Cp with respect to T. This is where the fitting came to play. Once I found the functional form of Cp, I used another package, SciPy, that could handle my integration. Below is all the code that did this:
I had found that this was precisely the application of fundamental skills I needed to take the next step. In completing this mini problem, I was learning how to effectively research documentation, utilize the (oh so) important functions in python, manipulate numbers, troubleshoot, and so many other things. I felt ready to begin the hyper-dual library!
Hyper-dual Number Library in Python
At this point in my progress, I had met with Dr. Capps twice since our first meeting and we both felt that I was ready to begin. She helped me get started and guided me in the right direction — starting with classes. To kick off this section, I include a journal entry from my very first hyper-dual library code!
Here, I am very simply replicating a desired use of this library. Obviously, Python doesn’t know how to handle hyper-dual numbers, so with this bit of code, I am allowing python to determine if one hyper-dual number is bigger than another.
I’ve defined some variables “qleft” and “qright” (standing for hyper-dual numbers on the left and right side of the operator) and I am asking the computer to determine if the left-side hyper-dual number is bigger than the right side hyper-dual number.
The first step of this process is to create a class — some “thing” that the computer doesn’t know about that I am teaching the computer to know about. This “thing” is “Hyperdual.” The first step is to tell the computer that a hyper-dual number will be presented to it with 4 ‘parts’: the real part, the first dual part, the second dual part, and the hyper-dual part. I’ve told the computer also how it can isolate each of those parts from any hyper-dual number with which it’s been presented.
After this administrative work, I needed to tell the computer how it can compare hyper-dual numbers based on size. I’ve told it that when given a hyper-dual number, “compare only the first ‘part’ (the real part) of each hyper-dual number — perform this comparison by seeing if the left is greater than the right (>).”
To show the results of the comparison, I’ve printed the function which returns a Boolean (true or false).
In the image shown, all of the highlighted matter is all that corresponds to the real ‘part’ of the hyper-dual numbers. You can see what the function is taking out of the hyper-dual numbers to compare them.
Quickly, I was able to copy & paste and find & replace so that the greater than function became the greater than function, the less than function, the greater than or equal to function, etc. I finished all the logical operators in the library. Now, effectively, one is “able to” compare two hyper-dual numbers!
This was only the tip of (the tip of) the iceberg, however. The list of operations in the Fortran library is extensive and quickly gets complicated.
I moved on next to the arithmetic operators (adding two hyper-dual numbers together for example) and already it got a challenging. One of the main aspects to adding hyper-dual numbers is that you add each respective part of one hyper-dual number to the other hyper-dual number. To make it clear, a hyper-dual number is shown like this:
$$
\begin{align*}
&x_{hd} = x_0+x_1 \epsilon_1+x_2 \epsilon_2+x_{12} \epsilon_1 \epsilon_2 \: ;\\\
&\text{where} \: x_1, \: x_2,\: \text{and}\: x_{12} \: \text{are real numbers} \: \text{and} \\
&\epsilon_1 \: \text{and} \: \epsilon_2 \: \text{are dual numbers}.
\end{align*}
$$
At first, I found it to be an issue that Python functions would only return one object, not independent objects that would each correspond to a part of the hyper-dual number sum. I learned later, however, that keeping the output of my functions inside 1 object is actually better. This is because it gives the computer and the user less to keep track of. Say someone wanted to use the addition function and right after wanted to use that sum and multiply it to another number. In this case, having a singular object as the sum is better. Ultimately, this is true for all cases, but it took me a while to realize this. The next hurdle to overcome was (and partly still is), exactly what object do I want these functions to output?
I went to Dr. Capps for guidance on this whole function output question, and, funny enough, I found out the answer was obvious. It’s important that the output of these arithmetic operations are objects within the “Hyperdual” class I constructed — the point of having a “Hyperdual” class is to formalize a way for the computer to handle these numbers after all. Arriving at this realization, the next step is to consider ways to achieve my functions becoming — succeeding in making this a reality is the big next step and where I am at on this project. The solution can either be class-based where I modify exactly how the “Hyperdual” class operates, or function based where I find a way for my functions to obey the class.
Another problem I’m facing is storing the “Hyperdual” objects into arrays, matrices, and tensors so that one hyper-dual number can be added to a certain set of others simultaneously. A solution to this problem undoubtedly will require iteration... so it’s good that I’ve practiced my loops! Nevertheless, I need to solve the aforementioned problem first and then I can move on to arrays.
Overall Reflection
My overall take away from the work I have completed so far is very much a positive one. I look back when I originally pitched this idea and I know that there are so many things that I can now do that I couldn’t then. I feel like the skills I have acquired put me in a spot where I am comfortable self-starting other difficult undertaking — specifically those having to do with Python.
My satisfaction however isn’t full... I will continue working on the Python hyper-dual library because I want to complete it and see it through. This “deliverable” is but part of the motivation I originally had to embark on this project, the rest is still standing. The plan was never to complete the hyper-dual library in 10 weeks, so I am not set back by this. But I do feel that I won’t be done until there is no longer more I can do more to build my tool-kit and make contributions to the topic I love.
I owe a gigantic thank you to Dr. Capps for her time and effort. Finding time to work on this project been very difficult despite the immense desire and interest I have for it. Knowing this, I can’t express how lucky I am to have worked with such a committed and generous mentor in Dr. Capps. She made time each weak to meet with me for an hour or more providing one on one guidance. Her direction and confidence in me have truly contributed to the success I have had in growing as a programmer and as a problem solver.
I would also like to thank Dr. Wood for his flexibility and openness for allowing me to work on something so original.
I am proud of the work I have done, and I’m looking ahead excitedly at the rest I still have unfinished.
Addendum
I am now a full year removed from the completion of this project. Happily, I can say that the skeleton of the Hyper-dual library is finished. It can be accessed here on GitHub (@atmmod is the GitHub for Dr. Shannon Capp's lab — here's her group's website).
I owe this whole project so much — this is including the paper I've written documenting the work which has promoted thoughtful reflection. Python has become the most used tool in my toolbox! Seemingly everyday I am writing/contributing a script for some new project, work task, or school assignment.
See the list of Python-related programs I've written!
