YC
Making Music and Art Through Machine Learning - Doug Eck of Magenta
@ Y Combinator
07/22/17
Takeaways
DE
Doug Eck
07/22/17
@ Y Combinator
The goal of the Magenta project is to create open-source tools and models that help creative people be even more creative through music and art generated by machine learning.
YC
Y Combinator Cast
07/22/17
@ Y Combinator
Whatever you now find weird, ugly, uncomfortable, and nasty about a new medium will surely become its signature, as modern art often reflects the sound of things going out of control and pushing a medium to its limits.
DE
Doug Eck
07/22/17
@ Y Combinator
When creating Magenta, I think about the division between engineering, research, and artistry, aiming to build interesting ways to make new kinds of art rather than just being a machine learning artist.
DE
Doug Eck
07/22/17
@ Y Combinator
We're currently working on a project called Henson, which aims to get deep learning models to generate new sounds, focusing on improving the quality and usability of the generated audio.
DE
Doug Eck
07/22/17
@ Y Combinator
We're rethinking our music sequence generation work, moving beyond simple models to care about expressive timing, dynamics, and polyphony, aiming to create high-quality music generation tools.
DE
Doug Eck
07/22/17
@ Y Combinator
We're gathering valuable feedback from artists using our tools, which is guiding us to improve the usability and fluidity of our music generation tools, especially for integration with software like Ableton.
DE
Doug Eck
07/22/17
@ Y Combinator
Artists have been sampling from our models and using them as a distance measure to explore creative possibilities, leading to a nice explosion of creativity in their work.
DE
Doug Eck
07/22/17
@ Y Combinator
The challenge is to make our models good enough and easy to use so that even non-expert musicians can find them genuinely musically engaging, focusing on generating interesting sounds rather than just melodies.
DE
Doug Eck
07/22/17
@ Y Combinator
Musicians with strong improvisational skills have been able to create interesting results using our primitive sequence generation tools, turning it into a collaborative game rather than just a tool.
DE
Doug Eck
07/22/17
@ Y Combinator
We haven't yet created a model that can generate an entire song because we haven't perfected the conditional aspects of the LSTM yet, which is crucial for handling long-term musical structures.
DE
Doug Eck
07/22/17
@ Y Combinator
To make music interesting, we need to incorporate hierarchical patterning and conditional models that can generate variations, such as chord changes, to create more engaging compositions.
DE
Doug Eck
07/22/17
@ Y Combinator
I don't believe that we made something that was better than popular music, but untrained listeners voted our tunes as sounding more appealing.
DE
Doug Eck
07/22/17
@ Y Combinator
These models are learning the principal axes of variance in music, identifying what's most important while having limited memory.
DE
Doug Eck
07/22/17
@ Y Combinator
When you sample from Sketch RNN with very low temperature, you get a more caricature-like output, resembling an average cat rather than a specific one.
DE
Doug Eck
07/22/17
@ Y Combinator
As Magenta progresses, I predict that machine learning will become a part of the toolkit for artistic expression, including music and art.
DE
Doug Eck
07/22/17
@ Y Combinator
Our job is to build smart AI tools, but we can't predict where this technology will go; it's essential to admit that uncertainty.
DE
Doug Eck
07/22/17
@ Y Combinator
Generative models might be able to create plots that are difficult for humans to generate but still make sense to readers.
DE
Doug Eck
07/22/17
@ Y Combinator
I can imagine that the right kind of language model could generate jokes that are super funny and have a unique flavor to them.
DE
Doug Eck
07/22/17
@ Y Combinator
The generative art world is evolving, and while some people may push back against it, it's okay if not everyone likes it.
DE
Doug Eck
07/22/17
@ Y Combinator
What's interesting about a project like Magenta is that it serves as a tool for creativity, not a replacement for human artists.
DE
Doug Eck
07/22/17
@ Y Combinator
If we presented this as a simple push-button solution for finished music, it would be a boring story.
DE
Doug Eck
07/22/17
@ Y Combinator
We want to create long-form pieces in music and art that make sense beyond just short segments, enhancing the creative process.
DE
Doug Eck
07/22/17
@ Y Combinator
The goal is to create tools that allow composers to offload interesting aspects of their work, like expressive timing and musical texture.
DE
Doug Eck
07/22/17
@ Y Combinator
To become a creative coder, visit our site at g.co/magenta, where you can find resources and contribute to our open-source projects.