Skip to content Skip to sidebar Skip to footer

Why Am I Able to Upload a Csv File to Matlab but Not Another

28. Mai 2018

How do you analyse Strava activities—such every bit runs or bike rides—with Colab, Python, Pandas, and Matplotlib? In this mail service, I am demonstrating how to get started, and will give you a taster of what is possible with this state-of-the-art technology for data assay.

I am both curious and skeptical. I honey running. And I also happen to like statistics. You lot won't exist suprised if I tell you now that I discover the following questions interesting:

  • Do I run longer distances on the weekend than during the calendar week?
  • Do I run faster on the weekend than during the week?
  • How many days do I residue compared to days I run?
  • How many kilometers did I cover during run-commutes?
  • Which month of the year practice I run the fastest?
  • Which calendar month of the year do I run the longest altitude?
  • Which quarter of the yr do I run the fastest?
  • Which quarter of the yr practise I run the longest altitude?
  • How has the average distance per run changed over the years?
  • How has the average time per run changed over the years?
  • How has my boilerplate speed per run inverse over the years?
  • How much altitude did I run in races and how much during training?
  • so on and and then on…

If you're bored by now, that's okay. Maybe you better end reading hither. Even so, should you be curious how to respond these questions past analysing sports activities with Colab, Python, Pandas, and Matplotlib, continue reading.

If you take not heard of Colab, Pandas, Matplotlib before—that's okay! You should still be able to get a picture of what these are capable of.

  • Strava: a mobile application that allows you lot to runway athletic activities. See www.strava.com.
  • Colab: a Python notebook highly suited for information assay using a "literal-programming" style. See colab.research.google.com.
  • Pandas: open-source library providing information structures and data assay tools for Python. See pandas.pydata.org.
  • Matplotlib: a second plotting library for Python. See matplotlib.org.

Strava provides various tables and visualizations on its website. But I notice that Strava rarely happens to visualize the data in a way that answers my questions. What to practice?

Outline of steps

The solution I describe here is to export all the activities recorded with Strava to a comma-separated values file (CSV)—runs, bike trips, hiking trips, and ice-skating trips. Then load the activities into Pandas data frames, and utilize these data frames to analyse and plot the activities.

  1. request all activities from Strava (instructions) as a naught archive
  2. import activities.csv from the zip archive into Google Sheets (instructions)
  3. create a new Colab (instructions)
  4. import information from Google Sheets into Colab (use the snippet "Importing data from Google Sheets")
  5. use Pandas to transform and process the activities
  6. use Matplotlib for creating visualizations

I am not going to provide a detailed walkthrough here—you can wait at the Colab.

Download Colab notebook

File: Analysing_Strava_activities_using_Colab,_Pandas_and_Matplotlib_(Part_1).ipynb [6.45 kB]
Category:
Download: 811

You tin open this Colab notebook using Get to File>Upload Notebook… in Colab.

Getting started

In the following, I volition explain some key steps in how to get started with analysing your Strava activities.

It is easy to load information from CSV files, Microsoft Excel spreadsheets, or Google Sheets into a Pandas information frame.

Equally the very showtime step, we tin have out all your data on Strava ( (instructions). In the zip archive, we will find an activities.csv file containing a listing of all your activities. Nosotros can then upload this CSV file to Google Drive and convert it into a Google Sheet. Then we tin insert a snippet into a Colab document, which loads the Google Sail into a Pandas data frame.

Alternatively, you could probably also adjust the sharing settings of the CSV file in Google Bulldoze to make the file readable by anyone with the link, and then utilize the pandas.read_csv function to import the data from the CSV file. Any works best for yous…

Preparing the data

Assuming that nosotros take loaded a Google Sheet into the variable worksheet, we tin then create a Pandas information frame.

                        import            pandas            as            pd            rows            =            worksheet.get_all_values() activities_raw            =            pd.DataFrame.from_records(rows[1:], columns=rows[0]);          

The information frame activities_raw holds the unprocessed information from the spreadsheet. At present follows a typical step, and that is transforming the data into a shape that makes it easy to do further data assay. Let us copy the raw data before modifying it simply then that nosotros tin can avoid reloading the data.

                        #@championship Index activities, convert data types, convert units            activities            =            activities_raw.copy()          

Not all of the columns in the exported Strava activities are relevant; we can easily retain but a few columns. In this case, the time at which the activity started, the blazon of the action ("Run", "Hike", "Ride", "IceSkate"), the altitude, the duration, whether the activeness was a commute to piece of work, and the proper noun of the activity.

                        # Project            activities            =            activities[[            'date',            'type',            'distance',            'elapsed_time',            'commute',            'proper name'            ]]          

Let united states of america explicitly convert the columns to the desired information types. This drastically simplifies further processing as we tin can now rely on Pandas doing the right thing when due east.k. summing the values in a column. In this instance, let us choose engagement and type as the (multi-)index.

                        # Set information types            activities['distance']            =            activities['distance'].astype(float) activities['elapsed_time']            =            activities['elapsed_time'].astype(float) activities['commute']            =            activities['commute'].apply(lambda            10:            one            if            x            ==            'True'            else            0)          

Permit us now convert the distance from meters to kilometers, and the elapsed time from seconds to minutes—this makes sense for me considering I observe minutes per kilometer intuitive as a measure out of step.

                        # Catechumen units            activities['distance']            =            activities['distance']            /            yard            activities['elapsed_time']            =            activities['elapsed_time']            /            60          

Finally, and this is a crucial step in Pandas, allow us define the index for the dataframe such that nosotros can easily address each activeness by engagement and blazon. This makes it very easy to group activities by date and/or past blazon. Then nosotros can prove statistics for running only, or compare activity types.

If you are unfamiliar with Pandas information frames, the index of a data frame is somewhat similar to the main fundamental of a tabular array in a relational database—although the alphabetize of a data frame does non accept to be unique.

                        # Index            activities['appointment']            =            pd.to_datetime(activities_raw['engagement']) activities.set_index(['date',            'blazon'], inplace=            True)          

Answering basic questions

Let's await at some of the information.

                        print(     activities[['distance',            'elapsed_time']]            .sample(5)            .sort_index()            .to_string(formatters={            'distance':            '{:.0f}km'            .format,            'elapsed_time':' {:.0f}min'            .format,       }))          

This shows v randomly selected rows from the data frame.

          altitude elapsed_time date                type                       2015-02-01 10:06:52 Hike     25km       419min 2015-04-06 ten:58:24 Run      11km        84min 2016-08-31 08:01:32 Ride      5km        22min 2016-x-05 eighteen:02:36 Ride      6km        29min 2017-10-07 08:01:57 Run      78km      1068min

Getting the data into an advisable format is crucial for farther analysis. We are now in a good shape. Permit's accept a look for which blazon of activity I have the virtually recordings, and in which activeness I have covered the virtually distance:

                        print(     activities['distance']            .groupby(level=            'blazon')            .agg(['count',            'sum'])            .sort_values(by=            'count', ascending=            Fake)            .to_string(formatters={'sum':' {:.0f}km'            .format}))          

This prints:

          count     sum blazon                    Run         255  3153km Ride        100  1526km Hike         17   465km IceSkate      three   146km

Permit us find out over which time menses these activities were recorded:

            first_date            =            activities.index.levels[0].min() last_date            =            activities.index.levels[0].max()            print('Activities recorded from                        %s                          to                        %due south            , total of                        %d                          days.'            %            (     first_date.strftime('%Y-%grand-            %d            '),     last_date.strftime('%Y-%chiliad-            %d            '),     (last_date            -            first_date).days))          

This prints:

Activities recorded from 2014-06-23 to 2018-05-26, total of 1433 days.

So almost 4 years of information! We need to be careful, though. The data shows just the activities that were recorded on Strava. I know that I have not ever recorded wheel rides, hikes, and ice-skating trips. In contrast, I have been diligently recording nigh all of my runs over the past years; for running, the least corporeality of data is missing.

What is next

You might rightly enquire. And then why use Colab, Python, Pandas, Matplotlib to perform this basic data assay? We could too just have used Google Sheets or Microsoft Excel to convert betwixt units, compute sums, create pivot tables and so on; this nigh probable would accept been far easier to practice. I am not going to compare betwixt the dissimilar tools for data analysis. In the next articles, I volition expect at the unanswered "research" questions stated to a higher place, and to answer these, we need to perform more complex analysis of the data—hither we volition reap the benefits of our investment to get the activities into the right format. This is where Colab, Pandas, and Matplotlib volition shine. Stay tuned.

Read the next article in this series

Articles in this series:

  • Part 1
  • Part 2
  • Part 3
  • Part 4

schmidtnurreacuot.blogspot.com

Source: https://www.athoughtabroad.com/2018/05/28/analysing-strava-activities-using-colab-pandas-matplotlib-part-1

Post a Comment for "Why Am I Able to Upload a Csv File to Matlab but Not Another"