rss.xml

<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/xsl" href="assets/xml/rss.xsl" media="all"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Python Project Night Challenges</title><link>https://chicagopython.github.io/</link><description>The Chicago Python User Group's coding workshops for Python Project Night.</description><atom:link href="https://chicagopython.github.io/rss.xml" rel="self" type="application/rss+xml"></atom:link><language>en</language><copyright>Contents © 2020 &lt;a href="mailto:chicago-sig-organizers@python.org"&gt;Chicago Python User Group&lt;/a&gt; 
&lt;a rel="license" href="https://www.gnu.org/licenses/gpl-3.0.en.html"&gt;
&lt;img alt="Gnu Public License version 3.0"
style="border-width:0;"
src="https://www.gnu.org/graphics/gplv3-with-text-84x42.png"&gt;&lt;/a&gt;</copyright><lastBuildDate>Thu, 16 Jan 2020 02:57:03 GMT</lastBuildDate><generator>Nikola (getnikola.com)</generator><docs>http://blogs.law.harvard.edu/tech/rss</docs><item><title>Fuzzy String Matching</title><link>https://chicagopython.github.io/posts/fuzzy-string-matching/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h3&gt;Overview&lt;/h3&gt;
&lt;p&gt;Data collection has and is rapidly expanding. However, data often isn’t submitted and/or collected without the required cleanliness or detail. At ChiPy we face the issue of trying to match Project Night attendees’ Meetup names with their legal names, which are needed for venue security. As a human it’s often easy to tell when names with slight variations match (Mike vs Michael; missing initials, etc), but trying to match hundreds of names one at a time is time consuming. Your job is to match the meetup and given names as accurately as possible using the fuzzy matching technique(s) of your choosing.&lt;/p&gt;
&lt;p&gt;Background reading:&lt;br&gt;
- &lt;a href="http://www.basistech.com/whitepapers/the-name-matching-you-need-EN.pdf"&gt;The Name Matching You Need: A Comparison of Name Matching Technologies&lt;/a&gt;&lt;br&gt;
- &lt;a href="https://medium.com/bcggamma/an-ensemble-approach-to-large-scale-fuzzy-name-matching-b3e3fa124e3c"&gt;An Ensemble Approach to Large-Scale Fuzzy Name Matching&lt;/a&gt;&lt;br&gt;
- &lt;a href="https://towardsdatascience.com/fuzzy-matching-at-scale-84f2bfd0c536"&gt;Fuzzy Matching at Scale&lt;/a&gt;  &lt;/p&gt;
&lt;p&gt;Some Python libraries you might want to use:&lt;br&gt;
- &lt;a href="https://pypi.org/project/fuzzywuzzy/"&gt;fuzzywuzzy&lt;/a&gt;&lt;br&gt;
- &lt;a href="https://pypi.org/project/textdistance/"&gt;textdistance&lt;/a&gt;&lt;br&gt;
- &lt;a href="https://github.com/Bergvca/string_grouper"&gt;string_grouper&lt;/a&gt;  &lt;/p&gt;
&lt;h3&gt;Setup&lt;/h3&gt;
&lt;p&gt;There is no existing repo for this project, and no requirements to install. All you need to start is the data, which can be downloaded &lt;a href="https://drive.google.com/file/d/1WtW89K43Rwxq5ZM8Dyryv5EQgkkauOCF/view?usp=sharing"&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;Feel free to work how you see fit. That said, we strongly recommend setting up a virtual environment.&lt;/p&gt;
&lt;p&gt;If you are using Linux or OS X, run the following to create a new virtualenv:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt;
&lt;span class="k"&gt;source&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;bin&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="n"&gt;requirements&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;txt&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;On Windows, instead run the following:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt;
&lt;span class="n"&gt;venv&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;Scripts&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="n"&gt;requirements&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;txt&lt;/span&gt;
&lt;/pre&gt;


&lt;h3&gt;So what should we do?&lt;/h3&gt;
&lt;p&gt;The dataset has three columns: 
- meetup_id: The unique Meetup identifier for each user.&lt;br&gt;
- meetup_name: The publicly available display name of the Meetup user.&lt;br&gt;
- given_names: The "actual" name of the attendee, as given as a form response via Meetup.  &lt;/p&gt;
&lt;p&gt;Each row in the dataset has the True matching name. In some cases, the meetup and given names match exactly, in some cases they don't. You won't need the meetup_id while actually attempting to match meetup and given names, but you can use it to validate your approach.&lt;/p&gt;
&lt;p&gt;Some things you might want to consider along the way:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Are all the names usable? Could a human uniquely identify matches?&lt;/li&gt;
&lt;li&gt;What patterns can be identified in the data we're working with?&lt;/li&gt;
&lt;li&gt;What is our true goal with matching? In other words, when evaluating our process' success, how do we balance ensuring someone has preregistered with not turning too many people away at the door? To that end, what's the right evaluation metric to choose?&lt;/li&gt;
&lt;li&gt;How might our approach differ if instead of a couple hundred names we have 10,000, a million, or even a billion names to match?&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;Hints (for if you're stuck)&lt;/h3&gt;
&lt;p&gt;One easy way to load the data is with pandas:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;    &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;pandas&lt;/span&gt; &lt;span class="kn"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;pd&lt;/span&gt;

    &lt;span class="n"&gt;read_kwargs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="s2"&gt;"header"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"index_col"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"skip_blank_lines"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"names"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"meetup_names"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"given_names"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pd&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;read_csv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"fuzzy_names.csv"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="n"&gt;read_kwargs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dropna&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;given_names&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"given_names"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;meetup_names&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"meetup_names"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;The Levenshtein algorithm is one of the more basic and popular algorithms for fuzzy string matching. It has a few useful Python implementations, but fuzzywuzzy is probably the most popular.&lt;/p&gt;
&lt;p&gt;Sklearn has modules dedicated to evaluation metrics. One very simple metric to evaluate how your matching is going is accuracy. Try starting with &lt;code&gt;from sklearn.metrics import accuracy_score&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Happy Developing!&lt;/p&gt;&lt;/div&gt;</description><guid>https://chicagopython.github.io/posts/fuzzy-string-matching/</guid><pubDate>Thu, 16 Jan 2020 11:00:00 GMT</pubDate></item><item><title>Build an API with Django REST Framework</title><link>https://chicagopython.github.io/posts/django-rest-framework/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h2&gt;Build an API with Django REST Framework&lt;/h2&gt;
&lt;h3&gt;Overview&lt;/h3&gt;
&lt;p&gt;For this project, we will be creating a functioning REST API. REST APIs can help distribute useful information via GET requests, as well as post and alter databases in a user friendly fashion.&lt;/p&gt;
&lt;p&gt;This project will revolve around using Django and Django's REST framework to build an API for the dataset of your choice. Django is a full web framework capable of handling both back and front end portions of a web app; and the Django team has created great resources to make setting up a Django app quick and easy.&lt;/p&gt;
&lt;p&gt;While the project is structured around Django, feel free to use flask instead, if you're more comfortable.&lt;/p&gt;
&lt;h3&gt;Environment Setup&lt;/h3&gt;
&lt;p&gt;To avoid bloating of your primary working environment, we strongly recommend creating a virtual environment. The requirements.txt file includes the required packages, and the included versions have been tested for our needs - use different versions at your own risk.&lt;/p&gt;
&lt;p&gt;We also strong recommend using &lt;a href="https://atom.io/"&gt;Atom&lt;/a&gt; or &lt;a href="https://www.sublimetext.com/3"&gt;Sublime Text&lt;/a&gt; as your text editor. This project has also NOT been tested using Jupyter Notebook, PyCharm,
Spider, or any other ide/text editor/programming environment.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;For this challenge you will need Python 3.7, pipenv, and git installed. If you're not familiar with pipenv, it's a packaing tool for Python that effectively replaced the pip+virtualenv+requirements.txt workflow. If you already have pip installed, the easiest way to install pipenv is with &lt;code&gt;pip install --user pipenv&lt;/code&gt;; however, a better way for Mac/Linux Homebrew users is to instead run &lt;code&gt;brew install pipenv&lt;/code&gt;. More options can be found &lt;a href="https://pipenv-fork.readthedocs.io/en/latest/install.html#installing-pipenv"&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The project is in the ChiPy project night repo. If you do not have the repository already, run &lt;/p&gt;
&lt;p&gt;&lt;code&gt;git clone https://github.com/chicagopython/CodingWorkshops.git&lt;/code&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Navigate to the folder for this challenge:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;cd CodingWorkshops/problems/webdev/django_rest_framework_api&lt;/code&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Run &lt;code&gt;pipenv install&lt;/code&gt;, which will install all of the libraries we have recommended for this exercise.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;After you've installed all of the libraries, run &lt;code&gt;pipenv shell&lt;/code&gt;, which will turn on a virtual environment running Python 3.7.&lt;/li&gt;
&lt;li&gt;To exit the pipenv shell when you are done, simply type &lt;code&gt;exit&lt;/code&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;Instructions&lt;/h3&gt;
&lt;h4&gt;Find a Database&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Before advancing, find a database that you wish to use for a REST API. It helps if the data is something you are interested in, but don't waste too much time on this part. &lt;a href="https://www.kaggle.com/tags/databases"&gt;Kaggle&lt;/a&gt; has a great selection of publicly available databases. If you are looking for something specific, Google has a stellar &lt;a href="https://toolbox.google.com/datasetsearch"&gt;database search&lt;/a&gt; feature.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4&gt;Create Your First App&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Create a Django app in a local directory of your choosing. Feel free to use the &lt;a href="https://chicagopython.github.io/posts/django-rest-framework/(https:/docs.djangoproject.com/en/2.2/intro/tutorial01/)"&gt;Django tutorial&lt;/a&gt; to accomplish this, but please don't call your app the standard Polls App. Create a unique application inside of your Django directory to handle your database and models. Make sure the application is configured in your settings.py file!&lt;/li&gt;
&lt;/ul&gt;
&lt;h4&gt;Create a Django Model&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Create a Django model custom to your database. Feel free to take liberties like creating relational databases for your models. The model field types should match the intended fields of your database. Make sure to migrate your Django model when you are finished!&lt;/li&gt;
&lt;/ul&gt;
&lt;h4&gt;Configure the REST framework&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Make sure you appropriately configure Django REST Framework in your settings.py file. If you forget this step, Django to recognize the add on.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4&gt;Serialization&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Before creating a url or view, serialize your data. This allows Django to render data into a JSON format. Make sure you designate the table (model) and fields (features) you wish to include in your REST API.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4&gt;Create a View&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Use the standard Django REST framework to create your Django view. Django REST framework allows you to interact with your API in both JSON and a preset interactive template. If you feel like going the extra mile, make your database queryable to gather the information you need.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4&gt;Designating a URL&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;Finally, designate url addresses where your page views can be found. Make sure to create a URL scheme that makes sense to how the intended user will interact with your API.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4&gt;Running your Server&lt;/h4&gt;
&lt;ul&gt;
&lt;li&gt;At this point it is time to test your API. This can be accomplished by the manage.py runserver command. Django's default location is localhost:8000/. From there, follow the naming scheme you created in your urls. Feel free to play with your API by using those filterable features you created!&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;Useful Weblinks&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Django Startup and Features&lt;/p&gt;
&lt;p&gt;https://docs.djangoproject.com/en/2.2/intro/tutorial01/&lt;/p&gt;
&lt;p&gt;https://docs.djangoproject.com/en/2.2/ref/applications/&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Django Models&lt;/p&gt;
&lt;p&gt;https://docs.djangoproject.com/en/2.2/ref/models/fields/&lt;/p&gt;
&lt;p&gt;https://docs.djangoproject.com/en/2.2/topics/db/models/#automatic-primary-key-fields&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Django REST Framework&lt;/p&gt;
&lt;p&gt;https://www.django-rest-framework.org/#installation&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Serialization&lt;/p&gt;
&lt;p&gt;https://www.django-rest-framework.org/api-guide/serializers/#modelserializer&lt;/p&gt;
&lt;p&gt;https://www.django-rest-framework.org/api-guide/serializers/#specifying-read-only-fields&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Views and URLS&lt;/p&gt;
&lt;p&gt;https://www.django-rest-framework.org/tutorial/quickstart/#views&lt;/p&gt;
&lt;p&gt;https://www.django-rest-framework.org/tutorial/quickstart/#urls&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;&lt;/div&gt;</description><guid>https://chicagopython.github.io/posts/django-rest-framework/</guid><pubDate>Thu, 21 Nov 2019 11:00:00 GMT</pubDate></item><item><title>Predict Home Credit Defaults</title><link>https://chicagopython.github.io/posts/home-credit-default-risk/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h3&gt;Overview&lt;/h3&gt;
&lt;p&gt;Many people struggle to get loans due to insufficient or non-existent credit histories. And, unfortunately, this population is often taken advantage of by untrustworthy lenders.&lt;/p&gt;
&lt;p&gt;Tonight's project examines a dataset from a real bank that focuses on lending to people with little or no credit history. Their goal is to ensure that clients capable of repayment are not rejected. You will explore the dataset and make predictions whether someone will default or not, based on their application for a loan.&lt;/p&gt;
&lt;h3&gt;Your Task&lt;/h3&gt;
&lt;p&gt;Your goal is to train a binary classification model on the data in &lt;code&gt;default_risk_train_data.csv&lt;/code&gt; that optimized area under the ROC curve between the predicted probability and the observed target. For each &lt;code&gt;SK_ID_CURR&lt;/code&gt; in &lt;code&gt;default_risk_train_data.csv&lt;/code&gt;, you must predict a probability for the TARGET variable. Your deliverable to the bank will be a CSV with predictions for each SK_ID_CURR in the test set.&lt;/p&gt;
&lt;h3&gt;Setup&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;For this challenge you will need Python 3.7, pipenv, and git installed. If you're not familiar with pipenv, it's a packaing tool for Python that effectively replaced the pip+virtualenv+requirements.txt workflow. If you already have pip installed, the easiest way to install pipenv is with &lt;code&gt;pip install --user pipenv&lt;/code&gt;; however, a better way for Mac/Linux Homebrew users is to instead run &lt;code&gt;brew install pipenv&lt;/code&gt;. More options can be found &lt;a href="https://pipenv-fork.readthedocs.io/en/latest/install.html#installing-pipenv"&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The project is in the ChiPy project night repo. If you do not have the repository already, run &lt;/p&gt;
&lt;p&gt;&lt;code&gt;git clone https://github.com/chicagopython/CodingWorkshops.git&lt;/code&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Navigate to the folder for this challenge:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;cd CodingWorkshops/problems/data_science/home_credit_default_risk&lt;/code&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Run &lt;code&gt;pipenv install&lt;/code&gt;, which will install all of the libraries we have recommended for this exercise.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;After you've installed all of the libraries, run &lt;code&gt;pipenv shell&lt;/code&gt;, which will turn on a virtual environment running Python 3.7.&lt;/li&gt;
&lt;li&gt;From within the shell, run &lt;code&gt;jupyter lab default_risk.ipynb&lt;/code&gt; to launch the pre-started notebook.&lt;/li&gt;
&lt;li&gt;To exit the pipenv shell when you are done, simply type &lt;code&gt;exit&lt;/code&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;What's in this repository?&lt;/h3&gt;
&lt;p&gt;There are three data files, one metadata file, and a jupyter notebook.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;default_risk_train_data.csv -- The data you will use to train your models. Includes all potential features and the target.&lt;/li&gt;
&lt;li&gt;default_risk_test_data.csv -- The data you will use to test your models. Includes all potential features, but NOT the target (which theoretically reflect unknown future default status).&lt;/li&gt;
&lt;li&gt;perfect_deliverable.csv -- The CSV with perfect predictions for each SK_ID_CURR in the test set. You should only use this at the very end to test the model and NEVER factor it into training your model. To prevent overfitting, you should test models sparingly. This is the same format the final deliverable should be submitted to the bank in.&lt;/li&gt;
&lt;li&gt;default_risk_column_descriptions.csv -- Descriptive metadata for the columns found in the train and test datasets.&lt;/li&gt;
&lt;li&gt;default_risk.ipynb -- The jupyer notebook where all coding should be completed, unless you opt to work in a different environment.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;This project is based on a Kaggle competition, with a subset of the data provided for the sake of download size. Note that this data has not been cleaned for you, and you should expect to deal with real world data issues, such as missing values, bad values, class imbalances, etc.&lt;/p&gt;
&lt;h3&gt;So what should we do?&lt;/h3&gt;
&lt;p&gt;To successfully complete this challenge, you'll need to:
    1. become an expert on the data,
    2. clean the data,
    3. engineer the features for your model(s),
    4. test/validate your models,
    5. generate the deliverable the bank expects.&lt;/p&gt;
&lt;p&gt;Here are some tips/questions to consider along the way:
- Identify which columns are numerical and which are categorical
- Which columns are missing values, and what should be done about the missing values?
- Which features are relevant and why?
- Which features might you want to remove?
- What new features might you create?
- How will you deal with categorical data (e.g. Label Encoding, One-Hot encoding, etc).
- Is there any class imbalance?
- What models will you try? sklearn has been installed in your environment; and &lt;a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html"&gt;linear regression&lt;/a&gt;, &lt;a href="https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression"&gt;logistic regression&lt;/a&gt;, and &lt;a href="https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier"&gt;random forest&lt;/a&gt; models have been imported in the given notebook. Feel free, however, to use the library/models of your choice.&lt;/p&gt;&lt;/div&gt;</description><guid>https://chicagopython.github.io/posts/home-credit-default-risk/</guid><pubDate>Thu, 17 Oct 2019 11:00:00 GMT</pubDate></item><item><title>Make a Game</title><link>https://chicagopython.github.io/posts/make-a-game/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h3&gt;Overview&lt;/h3&gt;
&lt;p&gt;For a long time, computer games made use of few, if any, graphics. Many of them were text based adventures that you could run directly on your command line. Some examples included:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Zork"&gt;Zork&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Adventureland&lt;/li&gt;
&lt;li&gt;&lt;a href="https://en.wikipedia.org/wiki/Dwarf_Fortress"&gt;Dwarf Fortress&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;and many others. Players would input their directions using words and the computer would return back what happened. &lt;/p&gt;
&lt;h3&gt;Your Task&lt;/h3&gt;
&lt;p&gt;Your task for this evening is to, working together, create something fun to play! Your group will take turns typing (in other words, one computer per group and only one person typing at a time) and helping to develop (offering ideas, thoughts on what to do next, etc.). It can be helpful to have another person with their computer open to research, but ultimately, this is a group effort! Everyone should have a chance to write code, offer suggestions, research libraries, etc. &lt;/p&gt;
&lt;h3&gt;Setup&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;You'll need one computer that your group will share that can install and run &lt;a href="https://pipenv-fork.readthedocs.io/en/latest/"&gt;Pipenv&lt;/a&gt;. While an OS-X or Linux machine will likely do the best for this step, a Windows machine will be able to do it as well. If you run into any challenges installing Pipenv, please ask for help!&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;The project is in the ChiPy project night repo. If you do not have the repository already, run &lt;/p&gt;
&lt;p&gt;&lt;code&gt;git clone https://github.com/chicagopython/CodingWorkshops.git&lt;/code&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Navigate to the folder for this challenge:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;cd CodingWorkshops/problems/py101/make_a_game&lt;/code&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Run &lt;code&gt;pipenv install&lt;/code&gt;, which will install all of the libraries we have recommended for this exercise.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;After you've installed all of the libraries, run &lt;code&gt;pipenv shell&lt;/code&gt;, which will turn on a virtual environment running Python 3.7.&lt;/li&gt;
&lt;li&gt;Run &lt;code&gt;python run.py&lt;/code&gt; to see the program in its current state or &lt;code&gt;pytest -vv&lt;/code&gt; to run all tests.&lt;/li&gt;
&lt;li&gt;If you make changes, this project uses a library called &lt;a href="https://github.com/psf/black"&gt;Black&lt;/a&gt; to automatically format the code for you (this known as a &lt;a href="https://en.wikipedia.org/wiki/Lint_(software)"&gt;linter&lt;/a&gt;. To run it, from the root of the directory, run &lt;code&gt;black .&lt;/code&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;What's in this repository?&lt;/h3&gt;
&lt;p&gt;In this repository is a basic shell of a game. This game sets up a &lt;code&gt;Player()&lt;/code&gt; which parrots back what the player writes to it until they decide to leave. Some of the key features here that you might want to use or modify or extend are:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;em&gt;Tests&lt;/em&gt; -- in the &lt;code&gt;tests/&lt;/code&gt; folder are a series of tests to make sure that the &lt;code&gt;Player()&lt;/code&gt; object continues to work as expected. As you add new functionality, you might want to practice &lt;a href="https://en.wikipedia.org/wiki/Test-driven_development"&gt;test-driven development&lt;/a&gt; to ensure that your code continues to work as you want it to!&lt;/li&gt;
&lt;li&gt;&lt;em&gt;run.py&lt;/em&gt; -- This is the main file that the player will run to play the game. One thing to note is the section that starts with &lt;code&gt;while player.in_game:&lt;/code&gt; -- this section sets up a loop that will keep running until the &lt;code&gt;in_game&lt;/code&gt; attribute is set to False. This way, your players can continue to do things and the game won't run once through the code and immediately finish. You'll likely add extra things into this section.&lt;/li&gt;
&lt;li&gt;&lt;em&gt;Player() class&lt;/em&gt; -- This class holds information about the player -- what its name is, what message it wants to repeat, whether it still wants to play the game...classes are useful for persisting or modifying some sort of collected state or values about a "thing", as well as defining actions that that thing may take. For example, our &lt;code&gt;Player()&lt;/code&gt; can currently &lt;code&gt;say_hello()&lt;/code&gt; and it has an &lt;code&gt;in_game&lt;/code&gt; status that can be either &lt;code&gt;True&lt;/code&gt; or &lt;code&gt;False&lt;/code&gt;. A different object might have different behaviors or different attributes that can be set. Depending on your game, you may want to set up more of these classes -- for example, you could set up a &lt;code&gt;Map()&lt;/code&gt; class to hold onto information about a map (what room the player is currently in, what rooms they can go to, etc.) or an &lt;code&gt;Enemy()&lt;/code&gt; class (what the enemy can do, how it interacts with the player, whether it is defeated or not, etc.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3&gt;So what should we do?&lt;/h3&gt;
&lt;p&gt;A good way to begin might be the following:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Decide what type of game you want to make: do you want to make a madlibs clone? Tic-tac-toe? A small dungeon? A word game? Put together a couple of ideas and identify what you'd like to build (and don't worry if you don't finish in time! This exercise is for you to be introduced to some Python concepts, not to emerge with a fully-developed game).&lt;/li&gt;
&lt;li&gt;Identify what basic building blocks you would need to interact with in the game. For example, if you were making a madlibs clone, you would want to identify what the user could enter, some scripts for those words to be entered into, and something that reads the story out after all the words have been entered. This can help with figuring out the basic flow of the game (for example, you would not want the story to be revealed before all the words are entered!)&lt;/li&gt;
&lt;li&gt;Start adding code and testing the game -- you could both add automated tests (like the ones in &lt;code&gt;tests/&lt;/code&gt; or try playing your game to see if it works. &lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Happy Developing!&lt;/p&gt;&lt;/div&gt;</description><guid>https://chicagopython.github.io/posts/make-a-game/</guid><pubDate>Thu, 19 Sep 2019 11:00:00 GMT</pubDate></item><item><title>GitHub Jobs API</title><link>https://chicagopython.github.io/posts/github-jobs-api/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h3&gt;Project Night Purpose&lt;/h3&gt;
&lt;p&gt;All of us need to look for a job at some point; and most every job board has its own API for users to post and pull data (though usually they charge money for access). The &lt;a href="https://jobs.github.com"&gt;GitHub Jobs page&lt;/a&gt; is a great simple API for a first look at this kind of data acqusition and analysis.&lt;/p&gt;
&lt;p&gt;Tonight's project uses the very popular Python HTTP library &lt;a href="https://2.python-requests.org/en/master/"&gt;requests&lt;/a&gt; along with &lt;a href="https://docs.python.org/3/library/json.html"&gt;json&lt;/a&gt; from the standard library. You will explore the data in the jobs API with the intent of learning something about the current job market for devs. There are a ton of caveats here: for example,&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Who chooses to post to GitHub Jobs? Are they representative of the overall population?&lt;/li&gt;
&lt;li&gt;How old are the postings?&lt;/li&gt;
&lt;li&gt;Is it safe to extrapolate statistics to the country? to Illinois? to Chicago?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Although the topic is a little serious we want to make sure you don't get discouraged by the data you pull (ask around--how many people in your group got their job from a GitHub posting?). The GitHub site shouldn't be taken as a good primary source for job availability or category...but that's part of data acquistition and analysis: assessing the strenghts and shortcomings of your data source. Possible discusison points for the group are things like:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Where might you get additional information?&lt;/li&gt;
&lt;li&gt;How often should you pull the data? What would capture over time give you?&lt;/li&gt;
&lt;li&gt;Does the dataset tell you anything about which companies use the GitHub for hiring at all? In which cities? Should we move to a different part of the country?&lt;/li&gt;
&lt;li&gt;Choose your own adventure and share with the group what you have learned!&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Most importantly - have fun! Accessing an API is a great core skill for data analysis and all experience is good experience. We hope that you are creative in your explanation and that each group discovers different things!&lt;/p&gt;
&lt;h3&gt;Setting up your environment&lt;/h3&gt;
&lt;p&gt;There is no pre-written code for this project, but we assume you have Python 3.+ installed on your machine.  If this is your fisrt project night, we recommend creating a folder for the project night repo: &lt;code&gt;mkdir chipy_projects &amp;amp;&amp;amp; cd chipy_projects&lt;/code&gt;. If you already have the project night repository on your machine, go to that directory and pull from master.&lt;/p&gt;
&lt;p&gt;If you are using Linux or OS X, run the following to create a new virtualenv:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;github_jobs_api&lt;/span&gt;
&lt;span class="k"&gt;source&lt;/span&gt; &lt;span class="n"&gt;github_jobs_api&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;bin&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;On Windows, run the following&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;github_jobs_api&lt;/span&gt; 
&lt;span class="n"&gt;github_jobs_api&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;Scripts&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;/pre&gt;


&lt;h3&gt;Getting the project&lt;/h3&gt;
&lt;p&gt;The project is in the ChiPy project night repo. If you do not have the repository already, run &lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;git&lt;/span&gt; &lt;span class="n"&gt;clone&lt;/span&gt; &lt;span class="n"&gt;https&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="o"&gt;//&lt;/span&gt;&lt;span class="n"&gt;github&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;com&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;chicagopython&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;CodingWorkshops&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;git&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Now we will:&lt;/p&gt;
&lt;p&gt;Go to the project:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;cd&lt;/span&gt; &lt;span class="n"&gt;CodingWorkshops&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;problems&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;data_science&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;github_jobs_api&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Install the packages we need into our environment:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="n"&gt;requirements&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;txt&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;view the &lt;em&gt;README.md&lt;/em&gt; file for additional information:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;cat&lt;/span&gt; &lt;span class="n"&gt;README&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;md&lt;/span&gt;
&lt;/pre&gt;


&lt;h3&gt;Have fun!&lt;/h3&gt;&lt;/div&gt;</description><category>api</category><category>EDA</category><guid>https://chicagopython.github.io/posts/github-jobs-api/</guid><pubDate>Thu, 15 Aug 2019 23:00:00 GMT</pubDate></item><item><title>Battery Life</title><link>https://chicagopython.github.io/posts/battery-life/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h2&gt;Predicting Battery Life - Challenge #1: Gathering Data&lt;/h2&gt;
&lt;p&gt;Portable electronics such as mobile phones and laptops have become a near necessity in our daily lives; and those devices share one essential resource in common: battery life. Have you ever sat on the floor to be by a power outlet for your laptop or cell phone? How about delayed leaving for an event because you had to make sure your phone was charged? In a perfect world, we wouldn't have to worry about battery life, but in the absence of the miracle battery, users must rely on indicators of remaining battery life.&lt;/p&gt;
&lt;p&gt;While there's still ongoing research into the capacity of batteries over time, the question of what percentage of charge remains has largely been solved in our everyday electronics. Most operating systems offer a way to display the percentage of battery life remaining. However, features that predict time remaining on the battery have been notoriously inaccurate, to the point where such features have been removed or hidden by default. Wouldn't it be nice if we could accurately predict when our phone was going to "die?"&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Your goal is going to be work toward that solution by gathering data to build a machine learning model predicting remaining battery life. You are tasked with determining what data we might want to collect for such a model, determining a strategy for ongoing collection of that data, actually collecting it, and organizing it into a form that will be usable to the machine-learning models of choice.&lt;/strong&gt; There are no right or wrong answer here, just things that are feasible and ultimately help drive better predictions.&lt;/p&gt;
&lt;p&gt;Before digging in, some background reading on how batteries work and what kinds of data/models can be useful is likely in order. Feel free to find your own resources, but here are a few:
- Overview of research and features: https://arxiv.org/pdf/1801.04069.pdf
- Battery Terminology: http://web.mit.edu/evt/summary_battery_specifications.pdf
- Battery Discharge Formulas: https://planetcalc.com/2283/&lt;/p&gt;
&lt;p&gt;Once you're ready to collect data, you'll likely want to collect running process and/or system utilization data as at least part of the data you collect. Gathering such data can vary drastically by hardware and operating system. To get you started, here are a few options to make extracting the data easier: 
- The &lt;a href="https://psutil.readthedocs.io/en/latest/"&gt;psutil&lt;/a&gt; library in python has cross-OS support, but only collects some such data.
- On Windows, there's the &lt;a href="http://timgolden.me.uk/python/wmi/tutorial.html"&gt;wmi&lt;/a&gt; library.
- On most distributions of Linux and MacOS, the standard librarys' os, sys, and subprocess modules can actually get you rolling pretty quickly, once you track down where system logs are stored!&lt;/p&gt;
&lt;p&gt;The rest is up to you, but some questions you might want to consider:
- When tracking battery/system data, how are you accounting for a device sometimes being plugged in?
- How will you account for different battery types, device types, and operating systems?
- Besides the obvious battery and system-related data, what features might help predict battery life?
- How can you collect enough data from enough sources to successfully  train a model?&lt;/p&gt;&lt;/div&gt;</description><guid>https://chicagopython.github.io/posts/battery-life/</guid><pubDate>Thu, 18 Jul 2019 11:00:00 GMT</pubDate></item><item><title>trackcoder</title><link>https://chicagopython.github.io/posts/trackcoder/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h2&gt;1. trackcoder: The Mentorship Journal&lt;/h2&gt;
&lt;p&gt;Chipy's mentorship program is an extra-ordinary jounery for becoming a better developer.
As a mentee, you are expected to do a lot - you read new articles/books, write code,
debug and troubleshoot, pair program with other mentees in coding workshop or your mentor.
This is involves managing time efficiently and doing the effective things.
But as the old adage goes, "you can't manage what you can't measure".&lt;/p&gt;
&lt;p&gt;This project is the first of the three part series of building tools for the mentees for
tracking time. The end goal of such a tool will be to give you a tool that helps you record mini
journal entires every day. The tool would also show you insight into your learning patterns
allowing you to make better decisions on when allocating time for self directed learnings beyond the
mentorship program.  &lt;/p&gt;
&lt;h3&gt;1.1. The Done list&lt;/h3&gt;
&lt;p&gt;Lets say you were to keep an account of every thing you have accomplished during your mentorship.
The minimalistic way for doing that would be noting things down in a text file - think of it as
a journal that you update frequently.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;    &lt;span class="nv"&gt;Date&lt;/span&gt;: &lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;
    &lt;span class="nv"&gt;first&lt;/span&gt; &lt;span class="nv"&gt;blog&lt;/span&gt; &lt;span class="nv"&gt;post&lt;/span&gt; &lt;span class="nv"&gt;completed&lt;/span&gt; 🏅
    &lt;span class="nv"&gt;learned&lt;/span&gt; &lt;span class="nv"&gt;about&lt;/span&gt; &lt;span class="nv"&gt;using&lt;/span&gt; &lt;span class="nv"&gt;click&lt;/span&gt; &lt;span class="nv"&gt;at&lt;/span&gt; &lt;span class="nv"&gt;project&lt;/span&gt; &lt;span class="nv"&gt;night&lt;/span&gt;
    &lt;span class="nv"&gt;updated&lt;/span&gt; &lt;span class="nv"&gt;cli&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="nv"&gt;the&lt;/span&gt; &lt;span class="nv"&gt;app&lt;/span&gt;

    &lt;span class="nv"&gt;Date&lt;/span&gt;: &lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;
    &lt;span class="nv"&gt;read&lt;/span&gt; &lt;span class="nv"&gt;good&lt;/span&gt; &lt;span class="nv"&gt;article&lt;/span&gt; &lt;span class="nv"&gt;on&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt; &lt;span class="nv"&gt;http&lt;/span&gt;:&lt;span class="o"&gt;//&lt;/span&gt;&lt;span class="nv"&gt;realpython&lt;/span&gt;.&lt;span class="nv"&gt;org&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;
    &lt;span class="nv"&gt;refactored&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;use&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="nv"&gt;orthogonal&lt;/span&gt; &lt;span class="nv"&gt;logic&lt;/span&gt;
    &lt;span class="nv"&gt;debugging&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt;
    &lt;span class="nv"&gt;met&lt;/span&gt; &lt;span class="nv"&gt;with&lt;/span&gt; &lt;span class="nv"&gt;mentor&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;fix&lt;/span&gt; &lt;span class="nv"&gt;decorator&lt;/span&gt; &lt;span class="nv"&gt;issues&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;With a little bit of effort, you can capture two more data points with each of the accomplishment
you are recording in this mini journal entry.&lt;/p&gt;
&lt;p&gt;(a) the time spent
(b) the type of activity&lt;/p&gt;
&lt;p&gt;These will form the basis of gaining insights into your personal learning patterns.&lt;/p&gt;
&lt;p&gt;For example:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="nv"&gt;Date&lt;/span&gt;: &lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;
&lt;span class="nv"&gt;first&lt;/span&gt; &lt;span class="nv"&gt;blog&lt;/span&gt; &lt;span class="nv"&gt;post&lt;/span&gt;, &lt;span class="nv"&gt;blogging&lt;/span&gt;, &lt;span class="mi"&gt;120&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;
&lt;span class="nv"&gt;learned&lt;/span&gt; &lt;span class="nv"&gt;about&lt;/span&gt; &lt;span class="nv"&gt;using&lt;/span&gt; &lt;span class="nv"&gt;click&lt;/span&gt; &lt;span class="nv"&gt;at&lt;/span&gt; &lt;span class="nv"&gt;project&lt;/span&gt; &lt;span class="nv"&gt;night&lt;/span&gt;, &lt;span class="nv"&gt;pair_programming&lt;/span&gt;, &lt;span class="mi"&gt;120&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;
&lt;span class="nv"&gt;updated&lt;/span&gt; &lt;span class="nv"&gt;cli&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="nv"&gt;the&lt;/span&gt; &lt;span class="nv"&gt;app&lt;/span&gt;, &lt;span class="nv"&gt;coding&lt;/span&gt;, &lt;span class="mi"&gt;20&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;

&lt;span class="nv"&gt;Date&lt;/span&gt;: &lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;
&lt;span class="nv"&gt;read&lt;/span&gt; &lt;span class="nv"&gt;good&lt;/span&gt; &lt;span class="nv"&gt;article&lt;/span&gt; &lt;span class="nv"&gt;on&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt; &lt;span class="nv"&gt;http&lt;/span&gt;:&lt;span class="o"&gt;//&lt;/span&gt;&lt;span class="nv"&gt;realpython&lt;/span&gt;.&lt;span class="nv"&gt;org&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;, &lt;span class="nv"&gt;research&lt;/span&gt;, &lt;span class="mi"&gt;45&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;
&lt;span class="nv"&gt;refactored&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;use&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="nv"&gt;orthogonal&lt;/span&gt; &lt;span class="nv"&gt;logic&lt;/span&gt;, &lt;span class="nv"&gt;coding&lt;/span&gt;, &lt;span class="mi"&gt;30&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;
&lt;span class="nv"&gt;debugging&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt;, &lt;span class="nv"&gt;debugging&lt;/span&gt;, &lt;span class="mi"&gt;30&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;
&lt;span class="nv"&gt;met&lt;/span&gt; &lt;span class="nv"&gt;with&lt;/span&gt; &lt;span class="nv"&gt;mentor&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;fix&lt;/span&gt; &lt;span class="nv"&gt;decorator&lt;/span&gt; &lt;span class="nv"&gt;issues&lt;/span&gt;, &lt;span class="nv"&gt;mentor&lt;/span&gt;, &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;While recording each of these activities have taken up time, you'll probably find one was
more useful than the other. So add another field to your
entry, effectiveness and refactor a little bit.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;, &lt;span class="nv"&gt;first&lt;/span&gt; &lt;span class="nv"&gt;blog&lt;/span&gt; &lt;span class="nv"&gt;post&lt;/span&gt;, &lt;span class="nv"&gt;blogging&lt;/span&gt;, &lt;span class="mi"&gt;120&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;, &lt;span class="mi"&gt;4&lt;/span&gt;
&lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;, &lt;span class="nv"&gt;learned&lt;/span&gt; &lt;span class="nv"&gt;about&lt;/span&gt; &lt;span class="nv"&gt;using&lt;/span&gt; &lt;span class="nv"&gt;click&lt;/span&gt; &lt;span class="nv"&gt;at&lt;/span&gt; &lt;span class="nv"&gt;project&lt;/span&gt; &lt;span class="nv"&gt;night&lt;/span&gt;, &lt;span class="nv"&gt;pair_programming&lt;/span&gt;, &lt;span class="mi"&gt;120&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;, &lt;span class="mi"&gt;4&lt;/span&gt;
&lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;, &lt;span class="nv"&gt;updated&lt;/span&gt; &lt;span class="nv"&gt;cli&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="nv"&gt;the&lt;/span&gt; &lt;span class="nv"&gt;app&lt;/span&gt;, &lt;span class="nv"&gt;coding&lt;/span&gt;, &lt;span class="mi"&gt;20&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;, &lt;span class="mi"&gt;3&lt;/span&gt;
&lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;, &lt;span class="nv"&gt;read&lt;/span&gt; &lt;span class="nv"&gt;good&lt;/span&gt; &lt;span class="nv"&gt;article&lt;/span&gt; &lt;span class="nv"&gt;on&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt; &lt;span class="nv"&gt;http&lt;/span&gt;:&lt;span class="o"&gt;//&lt;/span&gt;&lt;span class="nv"&gt;realpython&lt;/span&gt;.&lt;span class="nv"&gt;org&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;, &lt;span class="nv"&gt;research&lt;/span&gt;, &lt;span class="mi"&gt;45&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;, &lt;span class="mi"&gt;4&lt;/span&gt;
&lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;, &lt;span class="nv"&gt;refactored&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;use&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="nv"&gt;orthogonal&lt;/span&gt; &lt;span class="nv"&gt;logic&lt;/span&gt;, &lt;span class="nv"&gt;coding&lt;/span&gt;, &lt;span class="mi"&gt;30&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;, &lt;span class="mi"&gt;3&lt;/span&gt;
&lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;, &lt;span class="nv"&gt;debugging&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt;, &lt;span class="nv"&gt;debugging&lt;/span&gt;, &lt;span class="mi"&gt;30&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;, &lt;span class="mi"&gt;2&lt;/span&gt;
&lt;span class="mi"&gt;02&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;21&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="mi"&gt;2019&lt;/span&gt;, &lt;span class="nv"&gt;met&lt;/span&gt; &lt;span class="nv"&gt;with&lt;/span&gt; &lt;span class="nv"&gt;mentor&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;fix&lt;/span&gt; &lt;span class="nv"&gt;decorator&lt;/span&gt; &lt;span class="nv"&gt;issues&lt;/span&gt;, &lt;span class="nv"&gt;mentor&lt;/span&gt;, &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="nv"&gt;mins&lt;/span&gt;, &lt;span class="mi"&gt;5&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;If you save this file as a .csv now and open it in excel, you'll be able to get an account of how your time has been spent by selecting all the rows
for the last but one column. Taking the next step, you can very easily build a bar graph of time spent per day by including the first column.&lt;/p&gt;
&lt;p&gt;&lt;img alt="csv.png" src="https://chicagopython.github.io/images/csv.png"&gt;&lt;/p&gt;
&lt;p&gt;We could have stopped right here and let you continue with a spreadsheet. as a system for your mentorship journal. But lets make it a fun, reliable,and smooth.&lt;/p&gt;
&lt;h4&gt;1.1.1. The Data structure&lt;/h4&gt;
&lt;p&gt;Having looked at the data we intend to capture, lets try to reason how each entry in the row
can be represented by variables in a script and what their type would be like.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;    &lt;span class="n"&gt;task&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;str&lt;/span&gt;
    &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;str&lt;/span&gt;
    &lt;span class="k"&gt;timestamp&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;datetime&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;timestamp&lt;/span&gt;
    &lt;span class="n"&gt;mins&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;effective&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;
    &lt;span class="n"&gt;done&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;bool&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Note: I have sneaked in a variable &lt;code&gt;done&lt;/code&gt;, which can tell us if this task is completed. We would not use it right now, but with a little bit of effort we can use this field to enhance this app to have a Todo list feature. With these as a member variables, we can now define a &lt;code&gt;class&lt;/code&gt; for our &lt;code&gt;app.py&lt;/code&gt;. A class is nothing but an abstract representation of how each of our data record (or object) should look like and behave.&lt;/p&gt;
&lt;p&gt;You'll find this in &lt;code&gt;app.py&lt;/code&gt;-s line 16-22.&lt;/p&gt;
&lt;h4&gt;1.1.2. Replacing csv with a database&lt;/h4&gt;
&lt;p&gt;csv or spreadsheets are a good start for storing data, but are not without limitations. For project like ours, where we are looking to do automation, analytics and integration with more than one system we should be better of using a database.
There are a lot of databases to chose from, however but for something simple like ours we will use the &lt;code&gt;sqlite&lt;/code&gt; that comes with the python installation. Lets now look at the different ways how we can capture the data in a python script, persist it into the database, and as well as retrieve it for further processing.&lt;/p&gt;
&lt;h5&gt;1.1.2.1. Object Relational Mapper (ORM)&lt;/h5&gt;
&lt;p&gt;Each of your mini journal entry would be first captured as
a Python object using the &lt;code&gt;app.py&lt;/code&gt; script that we will build in part 1. &lt;code&gt;app.py&lt;/code&gt; will also convert the python object
into a database record using an Object Relatoinal Mapper(orm) and persist the data into the database. If you want to retrieve/update/delete a record from the database, the ORM allows you to use a similar Pythonic code to get the data back.&lt;/p&gt;
&lt;h5&gt;1.1.2.2. SQL/Pandas&lt;/h5&gt;
&lt;p&gt;While the ORM allows you to write Python directly, the primary mechanism for querying the data in relational databases is
Structed Query Language (SQL) which is english like query language that allows you to create,
retrieve, summarize and analyze the data. We would be using SQL and pandas in part three for analysis of the data.&lt;/p&gt;
&lt;h4&gt;1.1.3. Building the app in three parts&lt;/h4&gt;
&lt;p&gt;&lt;img alt="project.png" src="https://chicagopython.github.io/images/project.png"&gt;&lt;/p&gt;
&lt;h4&gt;1.1.4. Part 1: The terminal client&lt;/h4&gt;
&lt;p&gt;Since most of our development time is spent on a terminal, in part 1 we will be building a terminal based client. The client will provide a simple interface to add a data record of your mini journal entry into the database.&lt;/p&gt;
&lt;h4&gt;1.1.5. Part 2: Web interface&lt;/h4&gt;
&lt;p&gt;After the database has been populated with entries we made in part 1, we will build a web interface that pulls
up data from the database and presents a dashboard to show where time was spent.&lt;/p&gt;
&lt;p&gt;&lt;img alt="dashboard" src="https://chicagopython.github.io/images/dashboard.gif"&gt;&lt;/p&gt;
&lt;h4&gt;1.1.6. Part 3: Data Analysis&lt;/h4&gt;
&lt;p&gt;Once the data enty and output part is complete, in part 3, we will use data science tools to
answer questions regarding your learning patterns. As a strech goal, we will enhance the dashboard built in part 2, with metrics and insights coming out of part 3.&lt;/p&gt;
&lt;h3&gt;1.2. Part 1&lt;/h3&gt;
&lt;p&gt;In this project we will explore&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;How to build command line applications using &lt;code&gt;prompt_toolkit&lt;/code&gt;, &lt;code&gt;click&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;How to store data in sqlite database that comes with Python using &lt;code&gt;peewee&lt;/code&gt; ORM&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;1.3. Setup Instructions&lt;/h3&gt;
&lt;p&gt;You will need a text editor like Visual Studio Code, Atom or Sublime Text. Since this
you'll be working in a group, having an editor that does not get in the way of solving
the problem is essential. So stick to what every one in your team is familiar with.&lt;/p&gt;
&lt;h4&gt;1.3.1. Download .zip from github&lt;/h4&gt;
&lt;p&gt;If you are not familiar with &lt;code&gt;git&lt;/code&gt;, you can download the repository from &lt;a href="https://github.com/chicagopython/CodingWorkshops/archive/master.zip"&gt;here&lt;/a&gt;.
Clicking on the link will download a .zip file to your computer. Next you need to
navigate to the folder where it was downloaded and unzip the folder. Once you have
the CodingWorkshop directory, you can go to step 1.5.&lt;/p&gt;
&lt;h4&gt;1.3.2. Git and Github [Optional]&lt;/h4&gt;
&lt;p&gt;After completing the steps below you should have a github account and be able to push
your local changes to this repository to github.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Follow the setup steps described &lt;a href="https://help.github.com/articles/set-up-git/"&gt;here&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Read the steps described in &lt;a href="https://help.github.com/articles/fork-a-repo"&gt;fork a repo&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Use the steps described above to fork this repository &lt;a href="https://github.com/chicagopython/CodingWorkshops"&gt;CodingWorkshops&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The changes that you make as a part of this exercise, will be pushed to the fork you created for this
repository.&lt;/p&gt;
&lt;p&gt;In case you have already have created a fork of this repository in your github account, you will
want to bring it up to date with the recent changes. In that case,
you will need to do the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://help.github.com/articles/configuring-a-remote-for-a-fork/"&gt;configuring a remote fork&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://help.github.com/articles/syncing-a-fork/"&gt;syncing a fork&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;1.4. Python&lt;/h3&gt;
&lt;p&gt;This project has made no attempt to be compatible with Python 2.7. 😎&lt;/p&gt;
&lt;p&gt;Recommended version: Python 3.6 or higher.&lt;/p&gt;
&lt;h3&gt;1.5. Quick Git command refresher [Optional]&lt;/h3&gt;
&lt;p&gt;Below are the few most used git commands&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="nv"&gt;git&lt;/span&gt; &lt;span class="nv"&gt;checkout&lt;/span&gt; &lt;span class="nv"&gt;master&lt;/span&gt;          # &lt;span class="nv"&gt;checkout&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;master&lt;/span&gt; &lt;span class="nv"&gt;branch&lt;/span&gt;
&lt;span class="nv"&gt;git&lt;/span&gt; &lt;span class="nv"&gt;checkout&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nv"&gt;b&lt;/span&gt; &lt;span class="nv"&gt;feature&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="nv"&gt;cool&lt;/span&gt; # &lt;span class="nv"&gt;crate&lt;/span&gt; &lt;span class="nv"&gt;a&lt;/span&gt; &lt;span class="nv"&gt;new&lt;/span&gt; &lt;span class="nv"&gt;branch&lt;/span&gt; &lt;span class="nv"&gt;feature&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="nv"&gt;cool&lt;/span&gt;
&lt;span class="nv"&gt;git&lt;/span&gt; &lt;span class="nv"&gt;add&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nv"&gt;u&lt;/span&gt;                   # &lt;span class="nv"&gt;stage&lt;/span&gt; &lt;span class="nv"&gt;all&lt;/span&gt; &lt;span class="nv"&gt;the&lt;/span&gt; &lt;span class="nv"&gt;updates&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="nv"&gt;commit&lt;/span&gt;
&lt;span class="nv"&gt;git&lt;/span&gt; &lt;span class="nv"&gt;commit&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="nv"&gt;am&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Adding changes and commiting with a comment&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
&lt;span class="nv"&gt;git&lt;/span&gt; &lt;span class="nv"&gt;push&lt;/span&gt; &lt;span class="nv"&gt;origin&lt;/span&gt; &lt;span class="nv"&gt;master&lt;/span&gt;       # &lt;span class="nv"&gt;push&lt;/span&gt; &lt;span class="nv"&gt;commits&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;develop&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="nv"&gt;ci&lt;/span&gt; &lt;span class="nv"&gt;branch&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Note for this exercise, we will be working on the master branch directly. However,
that is NOT a best practice. Branches are cheap in git, so a new feature or fix
would first go to a branch, get tested, code reviewed and finally merged to master.&lt;/p&gt;
&lt;h3&gt;1.6. Documentation references&lt;/h3&gt;
&lt;p&gt;Below are the libraries used by this program.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="https://python-prompt-toolkit.readthedocs.io/en/master/"&gt;prompt_toolkit&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="http://click.pocoo.org/5/"&gt;click&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="http://docs.peewee-orm.com/en/latest/"&gt;peewee&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h3&gt;1.7. Exercise 0: Project Setup&lt;/h3&gt;
&lt;p&gt;After completing the steps in setup, you should have the cloned versoin of the fork of &lt;code&gt;CodingWorkshop&lt;/code&gt;
repository in your local machine. Lets take the time to look at the structure of this
project. All code is located under &lt;code&gt;/problems/py101/trackcoder&lt;/code&gt; directory. So from your
terminal go to the directory where you have cloned the repository.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;cd&lt;/span&gt; &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="k"&gt;to&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;CodingWorkshop&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;problems&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;py101&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;trackcoder&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Make sure you are in this directory for the remainder of this project.&lt;/p&gt;
&lt;p&gt;Run &lt;code&gt;pwd&lt;/code&gt; (&lt;code&gt;cwd&lt;/code&gt; for Windows) on the command prompt to find out which directory you
are on.&lt;/p&gt;
&lt;p&gt;Your output should end in &lt;code&gt;problems/py101/trackcoder&lt;/code&gt; and contain the files described
below.&lt;/p&gt;
&lt;h4&gt;1.7.1. &lt;code&gt;app.py&lt;/code&gt;&lt;/h4&gt;
&lt;p&gt;This file contains the code required to get you started with building the project.
You will be building on top of what has been provided in this file.&lt;/p&gt;
&lt;h4&gt;1.7.2. &lt;code&gt;Makefile&lt;/code&gt;&lt;/h4&gt;
&lt;p&gt;This file contains the commands that are required building the project.
You can run &lt;code&gt;make help&lt;/code&gt; to see what are the options.&lt;/p&gt;
&lt;p&gt;Note Makefile would not work on Windows out of the box.&lt;/p&gt;
&lt;h4&gt;1.7.3. &lt;code&gt;Pipfile&lt;/code&gt; and &lt;code&gt;Pipfile.lock&lt;/code&gt;&lt;/h4&gt;
&lt;p&gt;These two files are used by &lt;code&gt;pipenv&lt;/code&gt; to create a virtual enviornment that
isolates all the dependencies of this project from other python projects in your computer.
Learn more about &lt;a href="https://docs.pipenv.org/"&gt;pipenv&lt;/a&gt;.&lt;/p&gt;
&lt;h3&gt;1.8. Exercise 1: Build&lt;/h3&gt;
&lt;p&gt;From the &lt;code&gt;/problems/py101/trackcoder&lt;/code&gt; directory, run&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;make&lt;/span&gt;
&lt;/pre&gt;


&lt;ul&gt;
&lt;li&gt;Which packages got installed?&lt;/li&gt;
&lt;li&gt;Which version of python is getting used?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Skip this exercise for Windows. Install the dependencies using&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="n"&gt;prompt_toolkit&lt;/span&gt; &lt;span class="n"&gt;Click&lt;/span&gt; &lt;span class="n"&gt;peewee&lt;/span&gt;
&lt;/pre&gt;


&lt;h3&gt;1.9. Exercise 2: Run the program&lt;/h3&gt;
&lt;p&gt;First shell into your virtual environment&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;make&lt;/span&gt; &lt;span class="n"&gt;shell&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This should activate your virtual enviornment, i.e. give you access to a python
environment where all the dependencies for this project has been installed.&lt;/p&gt;
&lt;p&gt;Note: If the above command errors out, or you are on Windows, run the following to get into
a shell with the virtualenv acitvated.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;pipenv&lt;/span&gt; &lt;span class="n"&gt;shell&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;If everything fails,&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="n"&gt;prompt_toolkit&lt;/span&gt; &lt;span class="n"&gt;Click&lt;/span&gt; &lt;span class="n"&gt;peewee&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Start by running&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;py&lt;/span&gt; &lt;span class="c1"&gt;--help&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;What are the possible options that command has?
Run each option with --help option to see what is the help message provided.&lt;/p&gt;
&lt;h3&gt;1.10. Exercise 3: Fix the help message&lt;/h3&gt;
&lt;h4&gt;1.10.1. Interactive mode&lt;/h4&gt;
&lt;p&gt;Running &lt;code&gt;app.py&lt;/code&gt; with &lt;code&gt;-i&lt;/code&gt; - should start the app in interactive mode.
Once in interactive mode, there are two commands &lt;code&gt;add&lt;/code&gt; and &lt;code&gt;show&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;add&lt;/code&gt; command allows adding a new &lt;code&gt;Task&lt;/code&gt;. The format is&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c"&gt;% add b 10 first paragraph of first blog post&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;here &lt;code&gt;b&lt;/code&gt; is the abbreviations for blogging, &lt;code&gt;10&lt;/code&gt; shows the time taken for the task. Rest of the sentence is comment.
There are only 6 possible Task types&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;blogging (b)&lt;/li&gt;
&lt;li&gt;coding (c)&lt;/li&gt;
&lt;li&gt;debugging (d)&lt;/li&gt;
&lt;li&gt;pair programming at project night (p)&lt;/li&gt;
&lt;li&gt;research (r)&lt;/li&gt;
&lt;li&gt;meeting with mentor (m)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For example, an interactive session might look like&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c"&gt;% add b 10 first blog post&lt;/span&gt;
&lt;span class="c"&gt;% add c 10 finished cli&lt;/span&gt;
&lt;span class="c"&gt;% add d 120 debugging decorators&lt;/span&gt;
&lt;span class="c"&gt;% add m 120 always keep the final presentation in mind&lt;/span&gt;
&lt;span class="c"&gt;% add r 60 read articles on pandas&lt;/span&gt;
&lt;span class="c"&gt;% add p 120 learned about decorators&lt;/span&gt;
&lt;/pre&gt;


&lt;/blockquote&gt;
&lt;p&gt;The &lt;code&gt;show&lt;/code&gt; command allows listing of all the &lt;code&gt;Task&lt;/code&gt;-s added till now.&lt;/p&gt;
&lt;blockquote&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="o"&gt;%&lt;/span&gt; &lt;span class="k"&gt;show&lt;/span&gt;
&lt;span class="nv"&gt;b&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt; &lt;span class="nv"&gt;first&lt;/span&gt; &lt;span class="nv"&gt;blog&lt;/span&gt; &lt;span class="nv"&gt;post&lt;/span&gt;
&lt;span class="nv"&gt;c&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt; &lt;span class="nv"&gt;finished&lt;/span&gt; &lt;span class="nv"&gt;cli&lt;/span&gt;
&lt;span class="nv"&gt;d&lt;/span&gt; &lt;span class="mi"&gt;120&lt;/span&gt; &lt;span class="nv"&gt;debugging&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt;
&lt;span class="nv"&gt;m&lt;/span&gt; &lt;span class="mi"&gt;120&lt;/span&gt; &lt;span class="nv"&gt;always&lt;/span&gt; &lt;span class="nv"&gt;keep&lt;/span&gt; &lt;span class="nv"&gt;the&lt;/span&gt; &lt;span class="nv"&gt;final&lt;/span&gt; &lt;span class="nv"&gt;presentation&lt;/span&gt; &lt;span class="nv"&gt;in&lt;/span&gt; &lt;span class="nv"&gt;mind&lt;/span&gt;
&lt;span class="nv"&gt;r&lt;/span&gt; &lt;span class="mi"&gt;60&lt;/span&gt; &lt;span class="nv"&gt;read&lt;/span&gt; &lt;span class="nv"&gt;articles&lt;/span&gt; &lt;span class="nv"&gt;on&lt;/span&gt; &lt;span class="nv"&gt;pandas&lt;/span&gt;
&lt;span class="nv"&gt;p&lt;/span&gt; &lt;span class="mi"&gt;120&lt;/span&gt; &lt;span class="nv"&gt;learned&lt;/span&gt; &lt;span class="nv"&gt;about&lt;/span&gt; &lt;span class="nv"&gt;decorators&lt;/span&gt;
&lt;/pre&gt;


&lt;/blockquote&gt;
&lt;p&gt;For this exercise you need to update add helpful messages that will summarize what each
of the options for &lt;code&gt;app.py&lt;/code&gt; stand for.&lt;/p&gt;
&lt;h3&gt;1.11. Exercise 4: Run in interactive mode&lt;/h3&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;py&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Add some tasks and list them out by using the commands shown above. Play around with the up/down
arrow keys to access history of the commands.&lt;/p&gt;
&lt;p&gt;Exit the session using &lt;code&gt;ctrl+D&lt;/code&gt;. From your command prompt, run &lt;code&gt;ls -l&lt;/code&gt; in linux or mac or &lt;code&gt;dir&lt;/code&gt;
in windows. What is the name of the file that gets created?&lt;/p&gt;
&lt;p&gt;Using sqlite3&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;sqlite3&lt;/span&gt; &lt;span class="n"&gt;to_do_list&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt; &lt;span class="s1"&gt;'select * from ToDo;'&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Compare the output that you get from running &lt;code&gt;show&lt;/code&gt; and using the command above.&lt;/p&gt;
&lt;h3&gt;1.12. Exercise 4: Run in non-interactive mode&lt;/h3&gt;
&lt;p&gt;For ease of entering data the program can also be run in non-interactive mode&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;py&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt; &lt;span class="mi"&gt;30&lt;/span&gt;  &lt;span class="ss"&gt;"first blog post completed"&lt;/span&gt;
&lt;span class="n"&gt;python&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;py&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Add a few tasks that have been completed and list them non-interactively.
Note you'll need to put the description in quotes in this mode.&lt;/p&gt;
&lt;h4&gt;1.12.1. Optional: For non-windows users only&lt;/h4&gt;
&lt;p&gt;You can further simplify entering tracking your time by adding a bash shell alias.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;alias&lt;/span&gt; &lt;span class="n"&gt;add&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="err"&gt;'&lt;/span&gt;&lt;span class="n"&gt;function&lt;/span&gt; &lt;span class="n"&gt;_add&lt;/span&gt;&lt;span class="p"&gt;(){&lt;/span&gt; &lt;span class="n"&gt;python&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;py&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt; &lt;span class="s"&gt;"$@"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;&lt;span class="n"&gt;_add&lt;/span&gt;&lt;span class="err"&gt;'&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Then from your shell you can&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;$ add c &lt;span class="m"&gt;30&lt;/span&gt; &lt;span class="s2"&gt;"finished oauth"&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Add a similar shell alias for the &lt;code&gt;show&lt;/code&gt; command.&lt;/p&gt;
&lt;h3&gt;1.13. Exercise 5: Error handling&lt;/h3&gt;
&lt;p&gt;Currently we have two commands &lt;code&gt;add&lt;/code&gt; and &lt;code&gt;show&lt;/code&gt;. Lets say the user made a typo,
or was creative while trying to input a command.&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c"&gt;% add c api 30 complete&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;instead of&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c"&gt;% add c 30 api complete&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This results in the program crashing horribly with huge stack trace.
Add error handling to handle cases when the program is unable to &lt;code&gt;parse&lt;/code&gt; the input
passed by the user.&lt;/p&gt;
&lt;h3&gt;1.14. Exercise 6: Enhance the show command&lt;/h3&gt;
&lt;p&gt;Enhance the show command to summarize the output by task category.
Your summary should include how much time was spent on each of the task category.&lt;/p&gt;
&lt;p&gt;As seen above, we are using sqlite3. You may choose to do your summary calculation
using sql or write the logic in python.&lt;/p&gt;
&lt;h3&gt;1.15. Exercise 7: Add a field for task complete or not&lt;/h3&gt;
&lt;p&gt;Next take a look at&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="n"&gt;ToDo&lt;/span&gt;(&lt;span class="n"&gt;Model&lt;/span&gt;):
&lt;/pre&gt;


&lt;p&gt;This class has a list of fields - task, description, timestamp, mins, done.
Till now we have not been using this field. It has a default value of &lt;code&gt;True&lt;/code&gt;
to indicate that a completed task is being added.&lt;/p&gt;
&lt;p&gt;However, it might not always be the case. You might want to log your work,
and still have incomplete tasks. In fact, logging often and logging early is
encouraged! In order to faciliate that we need to optionally
take a fourth parameter in the input for adding a new task.&lt;/p&gt;
&lt;p&gt;Take a look at the decorator right above the &lt;code&gt;main&lt;/code&gt; function&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="nv"&gt;@click&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;option&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'--add'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;'-a'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;nargs&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;click&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;STRING&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nc"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;click&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;STRING&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="w"&gt;&lt;/span&gt;
&lt;span class="w"&gt;            &lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="k"&gt;None&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="w"&gt;&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;This is the starting point for allowing taking in an extra input.
You will find the relevant documentation &lt;a href="http://click.pocoo.org/5/options/"&gt;here&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Hint: Note the type of the field is boolean.
You will need to modify the &lt;code&gt;parse&lt;/code&gt;, &lt;code&gt;add&lt;/code&gt; and the &lt;code&gt;main&lt;/code&gt; function in order to complete
this exercise.&lt;/p&gt;
&lt;h3&gt;1.16. Exercise 8: Enhance the summary&lt;/h3&gt;
&lt;p&gt;Enhance your summary function to show how many tasks are in progress and how many are complete.
How you want to format the information is completely up to your choice.&lt;/p&gt;
&lt;h3&gt;1.17. Exercise 9: Hashtags&lt;/h3&gt;
&lt;p&gt;Now that you have enabled the flag to indicate if a task is complete or not, you
can log a much fine grained prorgress of your tasks. You can tag your task with
arbitary hashtags in order to provide better semantic information. For example:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c"&gt;% add p 120 #data_science learned about precision/recall&lt;/span&gt;
&lt;span class="c"&gt;% add b 120 finished the blogpost&lt;/span&gt;
&lt;span class="c"&gt;% add p 30 #data_science learned about roc curves&lt;/span&gt;
&lt;span class="c"&gt;% add p 30 #webdev added a flask interface&lt;/span&gt;
&lt;span class="c"&gt;% add d 30 #issues/7 found a bug, new github issue&lt;/span&gt;
&lt;span class="c"&gt;% add p 30 #issues/7 closed github issue 7&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Enhance the show command to optionally take a hashtag as parameter, that will filter out only
tasks which have that hashtag. Accrodingly your summary should reflect only
data relevant to that hashtag.&lt;/p&gt;
&lt;h3&gt;1.18. Exercise 10: Add a field for effectiveness&lt;/h3&gt;
&lt;p&gt;Next its time to add a score to your efforts. Add a field to the &lt;code&gt;ToDo&lt;/code&gt; class called effective,
where you can record how effective a task was. An effective score is a number between 1 to 5,
1 being the lowest and 5 being the highest.&lt;/p&gt;
&lt;p&gt;Armed with this data, you should be able to answer
(i) what is taking up most of your time?
(ii) which activities are the most effective for your growth&lt;/p&gt;&lt;/div&gt;</description><guid>https://chicagopython.github.io/posts/trackcoder/</guid><pubDate>Thu, 16 May 2019 04:48:53 GMT</pubDate></item><item><title>ChiPy Chipmunk Project Night </title><link>https://chicagopython.github.io/posts/chipy-chipmunks/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h3&gt;Project Night Purpose&lt;/h3&gt;
&lt;p&gt;Many people assume data scientists spend all day visualizing data and making impressive predictive models. While this isn’t untrue, the luckiest and most productive data scientists spend a lot of their time communicating. They communicate their model results - as well as their assumptions and limitations when making their models and doing analysis - in a way that is digestible to their stakeholders and colleagues. &lt;/p&gt;
&lt;p&gt;Tonight’s project is aimed towards that aspect of communication. You will be asked to make assumptions as a team - particularly as they pertain to this problem and what the stakeholders need. There are no exactly correct assumptions or answers for this project night. There may be assumptions and answers that clearly don’t have evidence to support them, but do not feel bogged down by getting the “right” answer.&lt;/p&gt;
&lt;p&gt;Most importantly - have fun. While this project night covers serious concepts, it is ridiculously silly and meant to be taken with a bit of lighthearted exploration and plenty of opportunities to make mistakes.&lt;/p&gt;
&lt;h3&gt;Setting up your environment&lt;/h3&gt;
&lt;p&gt;This project is contained in a jupyter notebook and is assuming you have Python 3.+ installed on your machine. If this is your fisrt project night, we recommend creating a folder for the project night repo: &lt;code&gt;mkdir chipy_projects &amp;amp;&amp;amp; cd chipy_projects&lt;/code&gt;. If you already have the project night repository on your machine, go to that directory and pull from master.&lt;/p&gt;
&lt;p&gt;If you are using Linux or OS X, run the following to create a new virtualenv:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;chipmunk&lt;/span&gt;
&lt;span class="k"&gt;source&lt;/span&gt; &lt;span class="n"&gt;chipmunk&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;bin&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;On Windows, run the following&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;chipmunk&lt;/span&gt; 
&lt;span class="n"&gt;chipmunk&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;Scripts&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;/pre&gt;


&lt;h3&gt;Getting the project&lt;/h3&gt;
&lt;p&gt;The project is in the ChiPy project night repo. If you do not have the repository already, run &lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;git&lt;/span&gt; &lt;span class="n"&gt;clone&lt;/span&gt; &lt;span class="n"&gt;https&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="o"&gt;//&lt;/span&gt;&lt;span class="n"&gt;github&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;com&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;chicagopython&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;CodingWorkshops&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;git&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Now we will:&lt;/p&gt;
&lt;p&gt;Go to the project:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;cd&lt;/span&gt; &lt;span class="n"&gt;CodingWorkshops&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;problems&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;data_science&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;chipmunks&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Install the packages we need into our environment:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;pip&lt;/span&gt; &lt;span class="n"&gt;install&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="n"&gt;requirements&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;txt&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Run our jupyter notebook server for the project:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;jupyter&lt;/span&gt; &lt;span class="n"&gt;notebook&lt;/span&gt;
&lt;/pre&gt;


&lt;h3&gt;Have fun!&lt;/h3&gt;&lt;/div&gt;</description><category>analytics</category><category>EDA</category><category>pandas</category><category>statistics</category><guid>https://chicagopython.github.io/posts/chipy-chipmunks/</guid><pubDate>Thu, 18 Apr 2019 23:00:00 GMT</pubDate></item><item><title>Chipmunks Data Science</title><link>https://chicagopython.github.io/posts/chipmunks-data-science/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h2 id="Project-Night-Purpose"&gt;Project Night Purpose&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Project-Night-Purpose"&gt;¶&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;Many people assume data scientists spend all day visualizing data and making impressive predictive models. While this isn’t untrue, the luckiest and most productive data scientists spend a lot of their time communicating. They communicate their model results - as well as their assumptions and limitations when making their models and doing analysis - in a way that is digestible to their stakeholders and colleagues.&lt;/p&gt;
&lt;p&gt;Tonight’s project is aimed towards that aspect of communication. You will be asked to make assumptions as a team - particularly as they pertain to this problem and what the stakeholders need. There are no exactly correct assumptions or answers for this project night. There may be assumptions and answers that clearly don’t have evidence to support them, but do not feel bogged down by getting the “right” answer.&lt;/p&gt;
&lt;p&gt;Most importantly - have fun. While this project night covers serious concepts, it is ridiculously silly and meant to be taken with a bit of lighthearted exploration and plenty of opportunities to make mistakes.&lt;/p&gt;
&lt;h2 id="Oh,-no!-We've-had-a-data-crash."&gt;Oh, no! We've had a data crash.&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Oh,-no!-We've-had-a-data-crash."&gt;¶&lt;/a&gt;&lt;/h2&gt;&lt;p&gt;As ChiPy leadership was preparing for &lt;a href="https://us.pycon.org/2019/"&gt;PyCon&lt;/a&gt; at the end of this month, they found that the dataset on our infamous &lt;em&gt;ChiPy chipmunks&lt;/em&gt; has disapeared. While they transition from Oracle to Postgres, the leadership team has enlisted your help as data scientists to analyze some salvaged chipmunk data. The PyCon organizers had a few questions about coding in Chicago, ChiPy, and chipmunks that need answers. We will get to those questions shortly, but first let's get to the data.&lt;/p&gt;
&lt;h3 id="Reading-in-the-Data"&gt;Reading in the Data&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Reading-in-the-Data"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;The salvaged chipmunk dataset is &lt;code&gt;chipmunk.csv&lt;/code&gt;. The wonderful &lt;a href="https://pandas.pydata.org/"&gt;pandas&lt;/a&gt; library, built on &lt;a href="http://www.numpy.org/"&gt;numpy&lt;/a&gt;, will let the team read in the data.&lt;/p&gt;
&lt;h4 style="color: #f92828;text-decoration: underline;"&gt;ChiPy Check-in&lt;/h4&gt;&lt;p&gt;Now is a good time to check in with the team. Is anyone familiar with &lt;code&gt;pandas&lt;/code&gt; and &lt;code&gt;numpy&lt;/code&gt;? Discuss with your team what these libraries are, what they allow data scientists to do, and then decide on what &lt;code&gt;pandas&lt;/code&gt; function will read in our data.&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h3 id="Setting-up-your-environment"&gt;Setting up your environment&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Setting-up-your-environment"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;This project is contained in a jupyter notebook and is assuming you have Python 3.+ installed on your machine. If this is your fisrt project night, we recommend creating a folder for the project night repo: &lt;code&gt;mkdir chipy_projects &amp;amp;&amp;amp; cd chipy_projects&lt;/code&gt;. If you already have the project night repository on your machine, go to that directory and pull from master.&lt;/p&gt;
&lt;p&gt;If you are using Linux or OS X, run the following to create a new virtualenv:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;python3 -m venv chipmunk
source chipmunk/bin/activate&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;On Windows, run the following&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;python3 -m venv chipmunk 
chipmunk\Scripts\activate&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id="Getting-the-project"&gt;Getting the project&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Getting-the-project"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;The project is in the ChiPy project night repo. If you do not have the repository already, run&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;git clone https://github.com/chicagopython/CodingWorkshops.git&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we will:&lt;/p&gt;
&lt;p&gt;Go to the project:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;cd CodingWorkshops/problems/data_science/chipmunks&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Install the packages we need into our environment:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;pip install -r requirements.txt&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Run our jupyter notebook server for the project:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;jupyter notebook&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The dataset is in the &lt;code&gt;csv&lt;/code&gt; file &lt;code&gt;chipmunks.csv&lt;/code&gt;.&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;np&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;pandas&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;pd&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;matplotlib.pyplot&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;plt&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;seaborn&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;sns&lt;/span&gt;
&lt;span class="o"&gt;%&lt;/span&gt;&lt;span class="k"&gt;matplotlib&lt;/span&gt; inline
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;# Read in the data&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h3 id="Exploring-the-data"&gt;Exploring the data&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Exploring-the-data"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;We need to be familiar with our data before we can answer questions about ChiPy and our chipmunks. Let's start with some questions we would ask of &lt;em&gt;any&lt;/em&gt; dataset:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;How many rows are in this dataset? What does each row represent?&lt;/li&gt;
&lt;li&gt;What does the data look like? Check the first 5 rows&lt;/li&gt;
&lt;li&gt;Is there missing data? If so, how much is missing?&lt;/li&gt;
&lt;li&gt;What columns are categorical?&lt;/li&gt;
&lt;li&gt;What are the unique number of observations for each column?&lt;/li&gt;
&lt;/ul&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Check the number of rows&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## See first 5 rows of data&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Check for missing data&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Check for categorical data and unique number of values&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h3 id="Was-there-missing-data?"&gt;Was there missing data?&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Was-there-missing-data?"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;We will keep exploring the data and start answering questions soon, but first let's address missing data (if there is any). What columns have missing data? What kind of data is missing?&lt;/p&gt;
&lt;h4 style="color: #f92828;text-decoration: underline;"&gt;ChiPy Check-in&lt;/h4&gt;&lt;p&gt;This a great point for discussion. If there is missing data - why might it be missing? Discuss some possible reasons with your team and decide on a reason that makes sense.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://en.wikipedia.org/wiki/Imputation_(statistics"&gt;Imputation&lt;/a&gt;) is the process of replacing missing data with some estimated value. The process can be as complicated (or simple) as you would like it to be! Given the possible reason for our missing data, what is an acceptable imputation?&lt;/p&gt;
&lt;p&gt;Impute any missing data in your dataset and note what assumptions you made as a team. If you are not sure how to replace data in &lt;code&gt;pandas&lt;/code&gt;, feel free to use google like a proper data scientist.&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;# Replace any missing data here&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;# Check your data for missing values to see if it worked!&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h2 id="Stakeholder-Questions"&gt;Stakeholder Questions&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Stakeholder-Questions"&gt;¶&lt;/a&gt;&lt;/h2&gt;&lt;h3 id="Question-#1"&gt;Question #1&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Question-#1"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;The great folks at PyCon want to know all about ChiPy and our chipmunks. They have heard that &lt;strong&gt;ChiPy is an inclusive and open community&lt;/strong&gt;. Can we support that claim with our data? Given that the &lt;code&gt;ChiPy&lt;/code&gt; column takes a value of &lt;code&gt;1&lt;/code&gt; for a ChiPy chipmunk and a value of &lt;code&gt;0&lt;/code&gt; for chipmunks not in ChiPy, start to explore this question.&lt;/p&gt;
&lt;p&gt;Some ideas to get you started:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Are chipmunks of different species represented in ChiPy?&lt;/li&gt;
&lt;li&gt;Are chipmunks of different sizes represented in ChiPy?&lt;/li&gt;
&lt;li&gt;Are chipmunks of different careers represented in ChiPy?&lt;/li&gt;
&lt;li&gt;Are spotted and not spotted chipmunks represented in ChiPy?&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 style="color: #f92828;text-decoration: underline;"&gt;ChiPy Check-in&lt;/h4&gt;&lt;p&gt;There are no right or wrong answers here, only well supported or poorly supported ones! Discuss as a group the aspects of the data you have looked at and if it constitutes enough evidence to justify an answer.&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Exploration of species&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Exploration of size&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Exploration of careers&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Exploration of spotted vs non-spotted&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h3 id="Question-#2"&gt;Question #2&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Question-#2"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;The word on the street at PyCon is that chipmunks that live in Chicago enjoy coding more than those that don't. Is this not true? Given that the &lt;code&gt;chicago&lt;/code&gt; column takes a value of &lt;code&gt;1&lt;/code&gt; for chipmunks that live in Chicago and a value of &lt;code&gt;0&lt;/code&gt; for chipmunks that do not, explore this question.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Visualize the distributions of &lt;code&gt;coding_enjoyment&lt;/code&gt; for chipmunks that do and do not live in Chicago.&lt;/li&gt;
&lt;li&gt;Come up with a way to test our question.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 style="color: #f92828;text-decoration: underline;"&gt;ChiPy Check-in&lt;/h4&gt;&lt;p&gt;Coming up with a proper way to test stakeholder questions can be an artform as well as a science. We have imported a few statistical tests below that may (or may not) be appropriate for our question. First consider a way to frame our question as something to &lt;em&gt;disprove&lt;/em&gt; (those familiar with jargon, let's construct a &lt;a href="https://en.wikipedia.org/wiki/Null_hypothesis"&gt;null hypothesis&lt;/a&gt;) - then conduct a test that may disprove it. Reading the documentation for the imported tests below may prove to be very helpful!&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;scipy.stats&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;ttest_ind&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;levene&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;chisquare&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Beautiful plot&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;## Statistical Test&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h3 id="Question-#2,-Continued"&gt;Question #2, Continued&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Question-#2,-Continued"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;We have now compared two groups of chipmunks - those that live in Chicago and those that do not - and have either rejected or failed to reject a null hypothesis. What values did the statistical test return and what do they mean? Can we be confident in our results? How confident?&lt;/p&gt;
&lt;p&gt;Regardless of our test results, what are the limitations of the test? One limitation is that we have information in our data that is &lt;em&gt;related&lt;/em&gt; to being in Chicago and might also have an effect on enjoyment of coding. &lt;a href="https://en.wikipedia.org/wiki/Regression_analysis"&gt;Regression analysis&lt;/a&gt; will allow us to examine the relationship between living in Chicago and enjoyment of coding while controlling for membership in ChiPy. Use the &lt;code&gt;statsmodels&lt;/code&gt; package to regress &lt;code&gt;chicago&lt;/code&gt; and &lt;code&gt;ChiPy&lt;/code&gt; on &lt;code&gt;coding_enjoyment&lt;/code&gt;. See &lt;a href="https://www.statsmodels.org/stable/index.html"&gt;this example&lt;/a&gt; for assistance.&lt;/p&gt;
&lt;h4 style="color: #f92828;text-decoration: underline;"&gt;ChiPy Check-in&lt;/h4&gt;&lt;p&gt;This regression model still has limitations, and there could be an entire project night on this task alone. What steps would need to be taken if we controlled for more characteristics of our data?&lt;/p&gt;
&lt;p&gt;This is also a good time to discuss what kind of information we are looking for in our regression model. What are coefficients and what do they mean? What is a p-value? Is it similar to a p-value from the statistical tests above?&lt;/p&gt;
&lt;p&gt;Lastly, modeling is fun, but don't forget the original question! Do chipmunks that live in Chicago enjoy coding more than those that don't?&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;statsmodels.api&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;sm&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;statsmodels.formula.api&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="nn"&gt;smf&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;# Regression model and summary&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h3 id="Question-#3"&gt;Question #3&lt;a class="anchor-link" href="https://chicagopython.github.io/posts/chipmunks-data-science/#Question-#3"&gt;¶&lt;/a&gt;&lt;/h3&gt;&lt;p&gt;ChiPy leadership wants to send 20 lucky ChiPy chipmunks to cheer the lovely folks at PyCon. However, it's unlikely that the data recovery efforts will be able to recover who is/isn't a member of ChiPy! ChiPy leadership has asked us to develop a predictive model to identify members as part of the process to allocate the 20 free tickets. To do this we will:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Make a train/test split to evaluate our model&lt;/li&gt;
&lt;li&gt;Scale our data&lt;/li&gt;
&lt;li&gt;Fit several models&lt;/li&gt;
&lt;li&gt;Decide on an evaluation metric&lt;/li&gt;
&lt;li&gt;Select this best model&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 style="color: #f92828;text-decoration: underline;"&gt;ChiPy Check-in&lt;/h4&gt;&lt;p&gt;The cell below transforms our data so that every feature (jargon for column) is numeric. Discuss with your team why this is could be an important step. Engineering features could also be an entire project night!&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;wide_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pd&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;get_dummies&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;df&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;drop_first&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;wide_data&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;head&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;sklearn.model_selection&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;train_test_split&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;sklearn.linear_model&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;LogisticRegression&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;sklearn.naive_bayes&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;BernoulliNB&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;sklearn.neighbors&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;KNeighborsClassifier&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;X_train&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;X_test&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y_train&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y_test&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;train_test_split&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;wide_data&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;drop&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'ChiPy'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;axis&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; 
                                                    &lt;span class="n"&gt;wide_data&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ChiPy&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; 
                                                    &lt;span class="n"&gt;test_size&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.33&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; 
                                                    &lt;span class="n"&gt;random_state&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;42&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;### Scale data&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;### Train models&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing text_cell rendered"&gt;&lt;div class="prompt input_prompt"&gt;
&lt;/div&gt;&lt;div class="inner_cell"&gt;
&lt;div class="text_cell_render border-box-sizing rendered_html"&gt;
&lt;h4 style="color: #f92828;text-decoration: underline;"&gt;ChiPy Check-in&lt;/h4&gt;&lt;p&gt;Having the proper evaluation metric is the most important process in predictive modeling. Below we have imported accuracy, precision, and recall. What are each of these metrics and when should they be used? Given that we want to give 20 PyCon tickets to only ChiPy chipmunks, which metric is most appropriate here? Black box evaluation methods like &lt;code&gt;classification_report&lt;/code&gt; will not be helpful here given the constraint of only having 20 tickets.&lt;/p&gt;

&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;sklearn.metrics&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;precision_score&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;recall_score&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;accuracy_score&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;confusion_matrix&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;### Get predictions...&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="c1"&gt;### Evaluate models, optimizing your predictions for 20 chipmunks!&lt;/span&gt;
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;
&lt;div class="cell border-box-sizing code_cell rendered"&gt;
&lt;div class="input"&gt;
&lt;div class="prompt input_prompt"&gt;In [ ]:&lt;/div&gt;
&lt;div class="inner_cell"&gt;
    &lt;div class="input_area"&gt;
&lt;div class=" highlight hl-ipython3"&gt;&lt;pre&gt;&lt;span&gt;&lt;/span&gt; 
&lt;/pre&gt;&lt;/div&gt;

    &lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;

&lt;/div&gt;&lt;/div&gt;</description><guid>https://chicagopython.github.io/posts/chipmunks-data-science/</guid><pubDate>Thu, 18 Apr 2019 03:39:06 GMT</pubDate></item><item><title>Flask App for Exchange Rates</title><link>https://chicagopython.github.io/posts/flask-app-for-exchange-rates/</link><dc:creator>Chicago Python User Group</dc:creator><description>&lt;div&gt;&lt;h2&gt;Flask App for Exchange Rates&lt;/h2&gt;
&lt;h3&gt;Objective&lt;/h3&gt;
&lt;p&gt;All developers in the modern day need to understand web technologies at some level. Whether you're interacting with a Jupyter notebook or querying a web api, understanding how a CLIENT requests information from a SERVER and to see how the SERVER produces its response is incredibly valuable.&lt;/p&gt;
&lt;h3&gt;Overview&lt;/h3&gt;
&lt;p&gt;We will make a web app that serves as both a CLIENT to an external api (exchangeratesapi.io). This app will show conversion rates for currencies, and then add some more complex data.&lt;/p&gt;
&lt;h3&gt;Prerequisites&lt;/h3&gt;
&lt;p&gt;For this project we recommend all use Atom (or Sublime) to write code and a shell/terminal to execute the program. All instructions will be given assuming a Python 3.6 install.&lt;/p&gt;
&lt;p&gt;You should probably have the &lt;a href="http://flask.pocoo.org/docs/1.0/quickstart/"&gt;Flask documentation&lt;/a&gt; up as we go through the exercise.&lt;/p&gt;
&lt;h3&gt;Initial Setup&lt;/h3&gt;
&lt;p&gt;Create a folder for this project: &lt;code&gt;mkdir mentorship_web &amp;amp;&amp;amp; cd mentorship_web&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;If you are using Linux or OS X, run the following to create a new virtualenv&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt;
&lt;span class="k"&gt;source&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;bin&lt;/span&gt;&lt;span class="o"&gt;/&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;On Windows, run the following&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;python3&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt; &lt;span class="n"&gt;venv&lt;/span&gt;
&lt;span class="n"&gt;venv&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;Scripts&lt;/span&gt;&lt;span class="err"&gt;\&lt;/span&gt;&lt;span class="n"&gt;activate&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Install Flask, our main web-app: &lt;code&gt;pip install flask&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;Create a new file called &lt;code&gt;app.py&lt;/code&gt;&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;flask&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Flask&lt;/span&gt;
&lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Flask&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="vm"&gt;__name__&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="nd"&gt;@app.route&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'/'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;hello_world&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"Hello World!"&lt;/span&gt;

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="vm"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="s1"&gt;'__main__'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;debug&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Run your flask app: &lt;code&gt;python app.py&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;It should display a link, paste that in your browser to see the running code.&lt;/p&gt;
&lt;h3&gt;Display some Exchange Rates&lt;/h3&gt;
&lt;p&gt;Define a dictionary with three entries like the following inside your hello route:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;exchange_rates&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="err"&gt;{&lt;/span&gt;
  &lt;span class="s1"&gt;'EUR'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'...'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s1"&gt;'GBP'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'...'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="err"&gt;'&lt;/span&gt; 
&lt;/pre&gt;


&lt;p&gt;Pick any currencies you want! We'll display the values as they convert to USD (because we are in Chicago). Better specify that now for clarity. Add this line just bellow your &lt;code&gt;app =&lt;/code&gt; definition, we'll use it later:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="n"&gt;BASE_CURRENCY&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'USD'&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Of course, fill in the &lt;code&gt;...&lt;/code&gt; with real currency values, otherwise our site is pointless! Use a tool like &lt;a href="https://www.x-rates.com/table/?from=USD&amp;amp;amount=1"&gt;X-Rates&lt;/a&gt; to look up the currency conversions.&lt;/p&gt;
&lt;p&gt;Now update the return string in the hello route to include the information about these currencies.&lt;/p&gt;
&lt;h3&gt;Make it Beautiful&lt;/h3&gt;
&lt;p&gt;Returning strings is fine...I suppose. But I want big, beautiful HTML! Research Flask's &lt;a href="http://flask.pocoo.org/docs/1.0/quickstart/#rendering-templates"&gt;&lt;code&gt;render_template&lt;/code&gt; function&lt;/a&gt; and add some beautiful looking HTML to format your. Hint: You'll want to create a &lt;code&gt;templates/&lt;/code&gt; folder in the same directory as your &lt;code&gt;app.py&lt;/code&gt;, and if you aren't using a kind of loop in your template...it's going to get real tough for you later! &lt;/p&gt;
&lt;p&gt;Test your work by adding another currency to your &lt;code&gt;exchange_rates&lt;/code&gt; dictionary and watch how your page changes.&lt;/p&gt;
&lt;h3&gt;Refactor&lt;/h3&gt;
&lt;p&gt;Define a function right under your &lt;code&gt;BASE_CURRENCY&lt;/code&gt; definition that looks like the following:&lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="nv"&gt;def&lt;/span&gt; &lt;span class="nv"&gt;exchange_rates&lt;/span&gt;&lt;span class="ss"&gt;()&lt;/span&gt;:
  &lt;span class="k"&gt;return&lt;/span&gt; # &lt;span class="nv"&gt;Move&lt;/span&gt; &lt;span class="nv"&gt;your&lt;/span&gt; &lt;span class="nv"&gt;dictionary&lt;/span&gt; &lt;span class="nv"&gt;here&lt;/span&gt;
&lt;/pre&gt;


&lt;p&gt;Then replace any references in your code to use the function instead of defining the dictionary in your route!&lt;/p&gt;
&lt;h3&gt;Automatic Data&lt;/h3&gt;
&lt;p&gt;Now comes the fun part. What if we could get the exchange rates on a live feed from a third party service? We can with &lt;a href="http://exchangeratesapi.io"&gt;ExchangeRatesAPI.io&lt;/a&gt;. This is a free api that runs over simple HTTPS! We'll have something like the following: &lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="nv"&gt;def&lt;/span&gt; &lt;span class="nv"&gt;exchange_rates&lt;/span&gt;&lt;span class="ss"&gt;()&lt;/span&gt;:
  &lt;span class="nv"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nv"&gt;requests&lt;/span&gt;.&lt;span class="nv"&gt;get&lt;/span&gt;&lt;span class="ss"&gt;(&lt;/span&gt;&lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;https://api.exchangeratesapi.io/latest&lt;/span&gt;&lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="ss"&gt;)&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; # &lt;span class="nv"&gt;What&lt;/span&gt; &lt;span class="nv"&gt;are&lt;/span&gt; &lt;span class="nv"&gt;we&lt;/span&gt; &lt;span class="nv"&gt;going&lt;/span&gt; &lt;span class="nv"&gt;to&lt;/span&gt; &lt;span class="nv"&gt;put&lt;/span&gt; &lt;span class="nv"&gt;here&lt;/span&gt; &lt;span class="nv"&gt;now&lt;/span&gt;? 
&lt;/pre&gt;


&lt;p&gt;Read up on &lt;a href="http://docs.python-requests.org/en/master/user/quickstart/#json-response-content"&gt;Requests' built in JSON Parsing&lt;/a&gt; and try to extract the data from the API response. Your end goal is to return all of the 
currencies from this function. &lt;/p&gt;
&lt;p&gt;Hint: This problem is a great time to use the Python debugger (&lt;a href="https://docs.python.org/3/library/pdb.html"&gt;&lt;code&gt;pdb&lt;/code&gt;&lt;/a&gt;)! Insert the following line before the &lt;code&gt;return&lt;/code&gt; command and interact with your server on the command line. So cool! &lt;/p&gt;
&lt;pre class="code literal-block"&gt;&lt;span&gt;&lt;/span&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;pdb&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;pdb&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;set_trace&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/pre&gt;


&lt;h4&gt;Correct Currency&lt;/h4&gt;
&lt;p&gt;Are we sure the previous step is getting us the currency in our &lt;code&gt;BASE_CURRENCY&lt;/code&gt; variable? Investigate the &lt;a href="http://exchangeratesapi.io"&gt;api docs&lt;/a&gt; to find out how you can change the base currency to the one we want.&lt;/p&gt;
&lt;h2&gt;You DID it! Now what?&lt;/h2&gt;
&lt;p&gt;From here on out we are trusting that you can use the documentation, look for resources on your own, and come up with clever solutions to these problems. Try each one, and if you get stuck, move on to a different one! Ask for help if you want more clarification or can't think of a way to do it. &lt;/p&gt;
&lt;h3&gt;Information Overload&lt;/h3&gt;
&lt;p&gt;Could we provide a form that would let people get only the currency they want? I don't need every currency. We'd probably use some kind of html form. And I bet Flask has some documentation on receiving requests from the client.&lt;/p&gt;
&lt;p&gt;e.g. Instead of &lt;code&gt;JPY-&amp;gt;*&lt;/code&gt; I only want to see &lt;code&gt;USD-&amp;gt;GBP&lt;/code&gt; or I only want to see &lt;code&gt;USD-&amp;gt;JPY&lt;/code&gt;.&lt;/p&gt;
&lt;h3&gt;Order by Value&lt;/h3&gt;
&lt;p&gt;How could we order the currencies by the value of the currency?&lt;/p&gt;
&lt;p&gt;e.g. If something is &lt;code&gt;.00001 USD&lt;/code&gt;, lets list that last, and if something is &lt;code&gt;987654321 USD&lt;/code&gt; lets list that first.&lt;/p&gt;
&lt;h3&gt;User-Specified Base Currency&lt;/h3&gt;
&lt;p&gt;Some users of our product have complained, rightfully, that they can only get the currency listed in &lt;code&gt;USD&lt;/code&gt;. We should let them specify what currency they want.&lt;/p&gt;
&lt;p&gt;e.g. Instead of &lt;code&gt;USD-&amp;gt;*&lt;/code&gt; I want to see &lt;code&gt;CNY-&amp;gt;*&lt;/code&gt;&lt;/p&gt;
&lt;h4&gt;Bonus: Links to each user-specified currency from the currency list&lt;/h4&gt;
&lt;p&gt;Wouldn't it be great if each currency as it appeared would link to its own currency conversion?&lt;/p&gt;
&lt;p&gt;e.g. From the home page I could click on &lt;code&gt;JPY&lt;/code&gt; to find out all of the conversions from &lt;code&gt;JPY-&amp;gt;*&lt;/code&gt;. &lt;/p&gt;
&lt;h3&gt;Caching&lt;/h3&gt;
&lt;p&gt;Do we really need to use an API request every time we do a call? How could we store the results of each run to avoid API abuse.&lt;/p&gt;&lt;/div&gt;</description><category>flask</category><guid>https://chicagopython.github.io/posts/flask-app-for-exchange-rates/</guid><pubDate>Thu, 21 Mar 2019 23:00:00 GMT</pubDate></item></channel></rss>