Music Analytics:
11 Jazz Standards

Kelly Curtis

02/15/2017

In a previous article, an introduction to how Curtis Cards use algorithms to generate metrics about a lead sheet is discussed. In this article, the discussion will focus on doing batch processing and analysis of multiple pieces to see how patterns stack up when the metrics are combined together.

Here is a snippet from a Curtis Card:

An extension to the basic Curtis Card is a summary of all the information the algorithms found, here is an example:

This is the summary of all the metrics the Curtis Card provides. This is where things start to get more interesting. By aggregating the results up to a piece level, it starts to show what you would need to know in order to master this particular given piece of music.

First:

You would need to know all the Chords that are listed in the Chord Metrics box.

Second:

You would need to know what chord types you would find at the beginning of a bar as shown in the Chord Symbol Strong Beats box.

Third:

You would need to know what chord types you would find at the back of a bar as shown in the Chord Symbol Weak Beats box.

Fourth:

You would need to know what actual chords you need to know as shown in the Root Chord Symbol Strong/Weak Beats box and where they show up in the musical bar.

Fifth:

The Chord Symbol Metrics tell you what chord types you really need to know and which ones are more for coloration versus the core sound palette of the piece.

Sixth:

It summarizes the patterns it found, such as a 5-1, 5-X (deceptive resolution), 2-5, 2-5-1 and what the final target chord is for the pattern.

In analytics, we call this technique map/reduce. Simply put, it is the process of mapping something first, and then reducing that information into an even smaller amount of information. It is in this way that massive amounts of information can be processed and further studied in an algorithmic fashion. It is a nice model for simplifying the complexity of musical pieces.

A quick note on a Strong and Weak beat:

A simple progression is shown where the chord changes on the 1st and 3rd beats of the bar. This is very common in Jazz music, and it is important to understand why it is being considered in study. Without diving into music theory, the general idea is that music alternates between strong harmonic pulses and weak harmonic pulses as it progresses through the piece. It is important as a performer to understand how these pulses affect the desired outcome of the piece of music.

Okay, getting dirty with the analytics. Some of the answers found may seem like common knowledge to some, but this kind of study is what should validate those assumptions.

Now that all the information in a lead sheet is processed into nice little size bits of information, doing analytics using standard tooling becomes easy. Using custom OLAP (online analytic processing) data structures, it is easy to create appropriate dimension and fact records for these pieces of music. Standard SQL queries allow for aggregation and statistical functions on the data set.

Since the data is now stored for one piece, what happens when we run all this data through multiple pieces? What do we find? Do certain patterns stack up more? Are there chord symbols that have more weight than others? All kinds of little questions can be posed of the data if analytics start to look at the data across multiple sources.

11 Lead Sheets were entered into the system; Curtis Cards were created; Metrics were saved centrally in a local data warehouse; Chaos ensued.

Those Lead Sheets are (taken from the Hal Leonard Real Book collection):

  1. Somebody Loves Me (key of F Major)
  2. Paper Doll (key of F Major)
  3. Lazy River (key of F Major)
  4. For Heaven's Sake (key of F Major)
  5. Beyond The Sea (key of F Major)
  6. Dancing On The Ceiling (key of F Major)
  7. Rosetta (key of F Major)
  8. Have You Met Miss Jones? (key of F Major)
  9. Georgia On My Mind (key of F Major)
  10. It's a Blue World (key of F Major)
  11. I Love You (key of F Major)

Why all in F Major? Time. Without building a transpose tool, normalizing before input was necessary, so all in F Major. To transpose is a simple function to move one piece of music to another key. This means that any music can be normalized to a standard key that can then be used to find more generalized patterns across the data set.

Why only 11? Again, time. As you read on, you will also find that 11 is plenty enough for patterns to start popping out that can be discussed. Also, to a performing musician, 11 pieces of music is easily 30 minutes of music.

Due to the bio-mechanical limitations of the human body for musical performance, simplifying the complexity of learning multiple pieces for a performer is paramount to their success.

Let us start with a simple question:

What happens when we simply count all the occurrences of a given symbol?

First:

501 total symbols

Second:

If we create a cutoff in the data at E7, or only looking at the chords with more than 10 occurrences in the set, there are 339, or 67%.

Third:

If we map those symbols to the their given function in the key of F, we can see they are all diatonic to the Key of F. Symbols not truly diatonic to F are A7 (V7/VI) and G7 (V7/V), which are called secondary dominants (V7/X).

Here they are in order of the key function:

IM7, IM6, II-7, III-7, IVM7, V7, VI-7, (V7/V), (V7/VI)

This gives us a great starting point to begin the educational process of teaching people how to read these symbols. If someone desires to master chord symbol based music (in this case all Major keyed pieces), they must master at least all the diatonic 7th chords in a given key. It will get them at least 67% of the way there to mastering the knowledge required for understanding this kind of music.

This is one of the fundamental use cases for our product the Chord Cycler.

Another question:

What happens when we simply count all the occurrences of a given symbol without a root?

Metrics:

394 Chords or 79% of all chords use the following 3 symbols:

Dominant 7, Minor 7, and Major 7 in that order.

This follows nicely with the 80/20 rule. If you want to be able to master this material, you can get there quicker by mastering the fundamental 3 chord types found:

R7, R-7, RM7

Next question:

What happens when we sum up chord symbols on there respectful beats in the bar?

First:

Probably the most interesting pattern that can be observed is that on strong harmonic pulses, it is primarily the chord types of R-7 and RM7

Of all 344 chord symbols that fell on the strong harmonic publse, 244 (79%) are of R-7 (or R-) and RM7 (or R)

Second:

On weak harmonic pulses, it is primarily the chord type R7

Of all 194 chord symbols that fell on the weak harmonic pulse, 137 (70%) are of R7

Within this set of pieces, it is easy to observe that at the front of the bar, it most likely going to be a RM7 or a R-7 and at the back of the bar, a R7 chord.

While other styles like Blues may rely on pure dominant function, this set of pieces alternate between RM7/R-7 and R7 chords. This can be useful information for those looking to master these pieces of Music.

Question:

What happens when we look at the patterns now and see what chords stand out?

5-1 pattern:

184 total 5-1 patterns found; 116 (63%) with a target chord of FM7, FM6, G-7, BbM7

2-5 pattern:

101 total 2-5 patterns found; 77 (76%) with a target chord of C7, D7, and G7

2-5-1 pattern:

87 total 2-5-1 patterns found; 59 (67%) with a target chord of FM7, FM6, G-7, BbM7

In the context of this set of pieces, it is interesting to see how common progression patterns end on a diatonic chord in their respective keys.

Very rarely does a full 2-5-1 pattern land on something outside of diatonic chords.

Knowing that V7, V7/V, and V7/II are really common is great for reducing complexity.

All kinds of actionable value for the Musician learning in this one.

Final question for this article:

What happens when we look at the melody notes and how they land in a given chord symbol (R-7: 1 b3, 5, b7); are there notes that have more weight than others?

Metrics:

951 melody notes total, with 771 (81%) falling on 1, 5, 9, b7, 3, 13, b3

This shows that 80% of the time, the great melodies from the era of these pieces relied on standard chord tones to generate their lines. These particular melody notes are across all chord types. It is safe to say that learning 1, b3, 3, 5, b7, 9, and 13 would give you enough material to create a structure to be used for composition or improvisation.

A quick note about the handwritten notes on all the charts:

Since all the pieces were normalized to a given major key (F), one can assign the function that chord has in the context of a major key (FM7 = IM7). This is useful for applying lessons learned from this to the context of any other Major key.

Here is a quick breakdown of all the chords that stood out in this analytics process:

FM6 = IM6

FM7 = IM7

G-7 = II-7

A-7 = III-7

BbM7 = IVM7

C7 = V7

D-7 = VI-7

D7 = V7/II

G7 = V7/V

A7 = V7/VI

Applying the basic concept of the 80/20 rule, it is safe to say that if one wants to master the style of these pieces, they need to focus on the list of chord symbols presented above to get 80% there for mastery. It also helps understand how diatonic chord progressions stay true to the given key in this style, and how it is about 20% special chord coloration.

This is great example of how a map/reduce process works. By mapping all the lead sheets, we were able to reduce the data into a format that allows for easier analytics. After all that work, the knowledge was reduced to a set of 10 chords. This kind of information is quite actionable to the Music community as people pursue their musical endeavors.

This is only the beginning of what data techniques can bring to the Music community. I hope you found the material interesting, and useful. Please feel free to email me if you have any questions or would like to discuss these topics further.

Thank you!

Kelly M. Curtis

bootstrap builder