#### Generative Music Programs as QR Codes --- They are among us. They speak in code. What do they say? What do they want?
---
--- # Broken Links
--- # Contingent Access
Note: QR codes are usually used as real-world links, physical anchor tags. Like like links, they do not *contain* the content; they merely point to it, and so they can break. Thus, access is contingent on the continued *availability* of the desired data and the host allowing you *access* to the data. Without these, the link is useless.
https://open.spotify.com/track/3pv7Q5v2dpdefwdWIvE7yH

--- # Can we put the content in the code? ---
# Related Work
## Barcoder #### Electronicos Fantasticos!
## Rewtro 
## WASM-4 
## Alternator 
## Alternator 
--- # Design
## Goals: 1. Contain music in a self-contained way - in the card itself - in a generic format - with minimal dependencies 2. Be scannable with stock camera apps - and look "scannable"
## Non-goals: 1. Contain arbitrarily large amounts of data - QR codes have a size limit (and cameras have limited resolution) - Use Alternator :-) 2. Build interactivity into card format - Consider JS + data URIs? `data:text/html,`
--- # Implementation
## Core pieces: - WebAssembly - AudioWorklet - QR codes
## The program on the card can: - Generate a sample (p for process) - Optionally, take a seed (s for setup) - Optionally, provide a description (d for description) - In this case, must also export m for memory via WebAssembly exports. No imports allowed! Note: Also, nothing outside the wasm. Even metadata is included in the wasm binary itself.
## From program to playback 
--- ## Reconciling design goals
# Problem ### Generated QR codes must 1. contain the program itself, and 2. decode to a URL to "just work" with stock camera apps
# Solution Store the program, in a URL, in a QR code
# New Problem Encoding overhead :-(
QR version 40, error correction L: 2953 bytes Base64, 6 bits in one character (8 bits): Loss of 739 bytes, down to 2214. ## Can we do better?
QR code modes: - Numeric (0-9) - Alphanumeric (0-9, A-Z, space, $, %, +, -, ., /, :) - Byte - Kanji/kana Ideal: decodes to valid URL, encodes compactly in QR code
# Solution Numeric mode: 3 digits in 10 bits. log2(10) / (10/3) = ~0.9966 Only loses 10 bytes!
# New New Problem iOS allergic to long URLs :-(
# Solution Alphanumeric mode:
2 characters (45 options) in 10 bits log2(45) / (10/2) = ~0.9985 But we can only use 43 of the characters in URLs. log2(43) / (10/2) = ~0.9866
 After URL prefix & base43 encoding,
2893 bytes remain for the program.
Sacrificed 2% of our space for usability. Note: for reference, that's a smaller percentage than the difference between a kilobyte (1000) and kibibyte (1024).
--- # Example
```text (module (func (export "p") (result f32) f32.const 0)) ```
Assembled as 37-byte binary: ```sh $ xxd minimal.wasm 00000000: 0061 736d 0100 0000 0105 0160 0001 7d03 .asm.......`..}. 00000010: 0201 0007 0501 0170 0000 0a09 0107 0043 .......p.......C 00000020: 0000 0000 0b ..... ```
Base-43: ```text 0LU+22:+M37ZMT5Q9VNHKG2IHPEYQS$DPRF8-N-CW7:5N3PBPNBI81 ``` with URL prefix: https://ijc8.me/s?c=0LU+22:+M37ZMT5Q9VNHKG2IHPEYQS$DPRF8-N-CW7:5N3PBPNBI81
--- # Composing Cards
## How much music can we make in under 3kb? Note: For context, that's about 16 milliseconds of audio. The score card generates as much data as the whole card contains in that time.
## Option 1: "Create" tab
## Option 2: anything else (recommended)
--- # Language Choice - Runtime must fit in the score card itself! - So, use a language with a minimal runtime. - Preference for low-level, compiled languages. - which can compile to wasm - How about C? (unintentional anagram: "C as Record") Note: Experiments with other languages. Faust may be promising. --- ## Composing Cards in C
```c #include
const char d[] = "noise example"; void s(unsigned int seed) { srand(seed); } float p() { return rand() / (float)RAND_MAX * 2 - 1; } ``` ```sh emcc -o noise.wasm -Oz noise.c --no-entry -sSUPPORT_ERRNO=0 \ -sSUPPORT_LONGJMP=0 -sEXPORTED_FUNCTIONS=_d,_s,_p ```
```c const char d[] = "bytebeat"; int sample = 0; float p() { int t = sample++ / 5; // Bytebeat expression: signed char x = t * (42 & t >> 10); return (float)x / 128; } ``` ```sh emcc -o bb.wasm -Oz bb.c --no-entry -sSUPPORT_ERRNO=0 \ -sSUPPORT_LONGJMP=0 -sEXPORTED_FUNCTIONS=_d,_p ```
--- # deck.h ## a little library
### for composing cards ---
```c #include "deck.h" card_title("FM thing"); float t, dur, start, end, phase; float mod_depth, mod_phase, mod_ratio; float dur_options[] = {0.25, 0.5, 1.0, 2.0, 4.0}; void setup(unsigned int seed) { srand(seed); dur = choice(dur_options); start = uniform(0, 1000); end = uniform(0, 1000); mod_ratio = uniform(1, 100); mod_depth = uniform(0, 1); } float process() { if (t > dur) return 0; float freq = ramp(t, dur, start, end); float mod_freq = freq / mod_ratio; freq *= 1 + mod_depth * sqr(&mod_phase, mod_freq); float out = sqr(&phase, freq) * env(t, dur); t += dt; return out; } ``` ```sh $ ./build.sh fm-ramp.c Before stripping: 663 After stripping: 497 ```
```c #include "deck.h" card_title("arpeggios"); setup_rand; const int chords[][5] = {{0, 2, 4, 6, 7}, {0, 2, 3, 5, 7}, {0, 1, 3, 6, 7}, {1, 3, 4, 6, 8}}; const int scale[] = {0, 2, 3, 5, 7, 9, 10, 12, 14}; typedef struct { uint8_t length; const uint8_t *order; } pattern_t; const pattern_t patterns[] = { {3, (uint8_t []){0, 1, 2}}, {3, (uint8_t []){2, 1, 0}}, {4, (uint8_t []){0, 1, 2, 3}}, {4, (uint8_t []){4, 3, 2, 1}}, {4, (uint8_t []){0, 1, 2, 4}}, {4, (uint8_t []){0, 2, 1, 2}}, {6, (uint8_t []){0, 1, 2, 3, 2, 1}}, {8, (uint8_t []){0, 1, 2, 3, 4, 3, 2, 1}}, }; typedef struct { int pitch; float dur; } event_t; event_t arp() { const float dur = 60.0 / 110; static int c, i, j, octave; static pattern_t pattern; gen_begin; for (;;) { octave = rand() % 3; pattern = choice(patterns); for (c = 0; c < SIZEOF(chords); c++) { for (i = 0; i < 4; i++) { for (j = 0; j < pattern.length; j++) { yield((event_t){scale[chords[c][pattern.order[j]]] + 48 + octave*12, dur / pattern.length}); } } } } gen_end((event_t){}); } float process() { static event_t event; static float t = 0, freq, phase; gen_begin; for (;;) { event = arp(); if (event.pitch == 0) { sleep(t, event.dur); continue; } freq = m2f(event.pitch); for (; t < event.dur; t += dt) { yield(saw(&phase, freq) * ad(t, 0.03, event.dur - 0.03)); } t -= event.dur; } gen_end(0); } ```
--- ## More examples
---
# Forbidden Techniques
```c // Tidalish patterns #include "deck.h" // Types typedef struct { double start; double end; } span_t; typedef struct { span_t span; int value; } event_t; typedef void pattern_func(void *self, span_t span); typedef struct { void *state; pattern_func *func; } pattern_t; void query(pattern_t pattern, span_t span) { pattern.func(pattern.state, span); } // Create object pool for events #define MAX_EVENTS 1024 event_t events[MAX_EVENTS]; int events_len = 0; void sort_events() { // heap sort (stdlib's qsort takes up too much space) int start = events_len / 2; int end = events_len; while (end > 1) { if (start > 0) { start--; } else { end--; event_t tmp = events[0]; events[0] = events[end]; events[end] = tmp; } int root = start; int child; while ((child = root * 2 + 1) < end) { if (child + 1 < end && events[child].span.start < events[child + 1].span.start) { child++; } if (events[root].span.start < events[child].span.start) { event_t tmp = events[root]; events[root] = events[child]; events[child] = tmp; root = child; } else { break; } } } } // atom - simplest pattern (one event that occupies an entire cycle) void atom(void *self, span_t span) { int start = truncf(span.start); int end = truncf(span.end); for (int i = start; i < end; i++) { events[events_len++] = (event_t){.span = {.start = i, .end = i + 1}, .value = (size_t)self}; } } // cat (AKA slowcat) - alternate between patterns, one per cycle typedef struct { pattern_t *patterns; int len; } patterns_t; void cat(void *_self, span_t span) { patterns_t *self = _self; int start = truncf(span.start); int end = truncf(span.end); for (int t = start; t < end; t++) { int index = t % self->len; int cycle = t / self->len; int start_event = events_len; query( self->patterns[index], (span_t){ .start = t == start ? (span.start - start + cycle) : cycle, .end = t == end - 1 ? (span.end - start + cycle) : (cycle + 1) } ); for (int i = start_event; i < events_len; i++) { events[i].span.start += (t - cycle); events[i].span.end += (t - cycle); } } } // stack - play patterns simultaneously, in parallel void stack(void *_self, span_t span) { patterns_t *self = _self; for (int i = 0; i < self->len; i++) { query(self->patterns[i], span); } } // fast - warp time within pattern typedef struct { pattern_t *pattern; double rate; } fast_t; void fast(void *_self, span_t span) { fast_t *self = _self; double rate = self->rate; int start_event = events_len; query(*(self->pattern), (span_t){span.start * rate, span.end * rate}); for (int i = start_event; i < events_len; i++) { events[i].span.start /= rate; events[i].span.end /= rate; } } // degrade - sometimes mute pattern void degrade(void *_self, span_t span) { pattern_t *pattern = _self; int start_event = events_len; query(*pattern, span); for (int i = start_event; i < events_len; i++) { // TODO: use a pure function of time instead of `rand()` if (rand() % 2) { // Remove event by moving the event at the end into its slot. events[i] = events[--events_len]; } } } double cps = 1; event_t ongoing_events[MAX_EVENTS]; int ongoing_events_len = 0; double t; #define AD(t, attack, decay) ((t) < (attack) ? ((t) / (attack)) : (1 - ((t) - (attack)) / (decay))) #define SAW(phase) ((phase) * 2 - 1) float synthesize_event(event_t event, double t) { double freq = m2f(event.value); t -= event.span.start / cps; double dur = (event.span.end - event.span.start) / cps; double phase = freq * t; phase = phase - trunc(phase); return AD(t, dur / 4, dur * 3 / 4) * SAW(phase); } float mix_events() { float out = 0; for (int j = 0; j < ongoing_events_len; j++) { while (t >= ongoing_events[j].span.end / cps) { // Event is finished; remove it by moving the event at the end into its slot. ongoing_events[j] = ongoing_events[--ongoing_events_len]; if (j >= ongoing_events_len) { // No more events. return out; } } out += synthesize_event(ongoing_events[j], t); } return out / 2; } // Better living through macros. #define COMBINE(fn, ...) {.func = fn, .state = &(patterns_t){ .len = SIZEOF(((pattern_t []){__VA_ARGS__})), .patterns = (pattern_t []){__VA_ARGS__}}} #define STACK(...) COMBINE(stack, __VA_ARGS__) #define CAT(...) COMBINE(cat, __VA_ARGS__) #define FAST(r, p) {.func = fast, .state = &(fast_t){ .rate = (r), .pattern = &(pattern_t)p}} #define DEGRADE(p) {.func = degrade, .state = &(pattern_t)p} #define NOTE(p) {.func = atom, .state = (void *)(p)} #define FASTCAT(...) FAST(SIZEOF(((pattern_t []){__VA_ARGS__})), CAT(__VA_ARGS__)) // Set up pattern. // (Let's pretend this is Lisp!) pattern_t pattern = STACK( NOTE(48), DEGRADE( FASTCAT( NOTE(60), NOTE(67), NOTE(63), NOTE(67)))); setup_rand; float process() { static int cycle = 0, i; gen_begin; for (;;) { // Query our pattern to get the events for the current cycle. query(pattern, (span_t){cycle, cycle + 1}); sort_events(); for (int i = 0; i < events_len; i++) { debug("%d-%d: %f %f %d\n", cycle, i, events[i].span.start, events[i].span.end, events[i].value); } for (i = 0; i < events_len; i++) { // Skip events whose onset is not in this cycle. if (events[i].span.start < cycle || events[i].span.start >= (cycle + 1)) continue; // Generate more samples until event start. for (; t < events[i].span.start / cps; t += dt) { yield(mix_events()); } // Insert new event at the end. ongoing_events[ongoing_events_len++] = events[i]; } // Done processing events: generate more samples until the end of this cycle. for (; t < (cycle + 1) / cps; t += dt) { yield(mix_events()); } // Reset events and advance cycle. events_len = 0; cycle++; } gen_end(0); } ```
```c // Decode audio and play audio compressed with linear predictive coding // See https://ccrma.stanford.edu/~hskim08/lpc/ #include "deck.h" card_title("voice"); #define LPC_SAMPLE_RATE (8820) #define NW (264) // window size (30ms) #define ORDER (5) // order of LPC filter (number of coefficients per frame) #define COUNT (134) // number of frames const float Gscale = 0.00068916486550117879; const float Ascale = 5.363421; const int Fscale = 2048; // To further save space, we store this data as 16-bit ints rather than 32-bit floats. // Amplitudes (TODO: store as unsigned short for greater resolution) const short G[COUNT] = {1,7,18,3,2,14,60,138,343,496,620,1078,7691,25810,20165,7662,3059,2737,4003,2187,1127,947,679,531,502,708,628,830,2253,3630,419,63,321,167,816,323,1430,4349,5663,5826,1157,989,893,388,110,28,10,261,1731,2108,1453,2934,9984,11780,7177,3418,3484,2285,1749,1793,1185,1241,990,576,1035,3196,2542,2816,996,570,719,696,2240,4558,4768,1769,667,505,578,842,1279,2738,7197,20484,10776,4837,2375,1169,1853,7509,32767,19732,13679,5588,898,188,60,33,16,345,2074,1138,785,600,454,152,256,1252,2395,2990,910,351,368,535,1231,1110,956,957,119,45,13,7,104,93,142,62,66,33,20,17,6,6,5,6}; // Filter coefficients const short A[ORDER][COUNT] = { {4151,10246,13037,7441,5940,14079,16411,16732,16974,17663,16570,15744,14981,13753,13717,15261,15646,11032,5030,5445,6676,5883,4135,3586,2132,-298,-1950,-1841,-2431,-1974,50,-1433,5155,7644,14202,14612,16402,16794,16861,15635,18795,18454,17426,14291,15192,14944,14251,12636,12181,12625,12738,16734,18099,18571,19043,20819,20236,20350,20161,18874,20224,19542,18444,16421,16146,14825,14432,13684,15233,12381,8794,4376,-1122,-4101,857,4705,6530,9293,13132,16602,18922,18720,15785,13464,15209,17216,18128,18246,18021,16339,14226,13994,12249,11858,14053,13959,11789,11092,10450,14077,13054,12762,11067,10195,10321,10804,16712,17212,16279,16260,18996,18397,17787,18056,17683,17410,16972,15969,13758,10391,11019,11515,16339,15880,13694,13318,11655,10072,8286,8024,9740,9361,9391,9896}, {-6,-5471,-10793,-859,-341,-9721,-16798,-18864,-19325,-22476,-19180,-18701,-18075,-15582,-15194,-17849,-18090,-8166,-498,-1381,-2184,-1140,1142,1504,3386,4328,3763,3087,124,137,2329,2907,-4039,-1648,-9389,-9459,-15547,-16517,-16701,-13455,-21504,-20461,-17956,-10902,-13111,-12173,-9953,-11867,-9846,-10162,-11694,-20872,-25563,-26583,-27473,-32768,-30836,-31047,-30927,-27208,-31376,-29757,-26779,-20931,-20070,-17677,-16509,-14562,-17957,-10073,-2679,3000,4883,-747,-3337,-3536,-4253,-5023,-8670,-16707,-25537,-27305,-20045,-14726,-18275,-22512,-23883,-22912,-22905,-20321,-16856,-16287,-12614,-11531,-14806,-14577,-9922,-8882,-7473,-13435,-10099,-9629,-8400,-7996,-7248,-8198,-16908,-18455,-15841,-15474,-22906,-19081,-17187,-19217,-19496,-18744,-17818,-14807,-10185,-4930,-4929,-4627,-17989,-16504,-13165,-10885,-7825,-5494,-3050,-3649,-2977,-2730,-3640,-3602}, {268,1414,5219,456,769,665,9018,13085,12592,17937,13936,16018,15178,12724,13401,15209,15245,8477,4414,2819,1482,655,-853,-1340,-2130,301,967,-755,-3665,-2990,-744,-1261,-3348,-5676,-5292,-5531,4663,4909,4779,1782,8793,7280,5360,721,810,-800,-1770,4494,847,976,4979,14926,21106,21658,21904,28948,26223,26397,27266,22870,28835,27653,24440,18591,17587,15192,13638,11023,14535,5370,884,-533,2758,4092,1911,1720,1134,926,1264,5575,18496,24141,16177,11278,14796,18561,18540,16144,16727,16272,14007,13264,11185,10607,12279,11517,7142,5548,4136,4017,-627,-196,1399,2702,889,2656,7186,9424,6648,5494,12773,5200,3005,7357,9661,8652,8350,4327,2138,3833,1545,-1025,11932,9913,10904,5778,3531,3734,1751,2946,-1344,-1177,672,-923}, {88,-645,-1797,-1129,-191,837,-3660,-6871,-5488,-9476,-7000,-9656,-7729,-6157,-8146,-9189,-9689,-7727,-957,-846,-587,-296,-45,301,-331,-1632,-1791,-159,956,849,830,1719,2152,4392,9830,9625,1081,1814,2292,3244,1514,2786,3114,3529,6145,7536,5389,827,3987,4362,432,-5906,-10003,-9855,-9361,-14196,-12211,-12249,-13593,-10973,-15467,-15291,-13311,-10806,-10606,-8531,-7256,-5018,-7536,-1205,-697,1681,585,3141,90,-22,-1373,111,-134,2088,-6552,-12096,-6792,-4435,-7085,-9453,-8612,-6980,-7492,-8463,-7387,-6329,-6206,-6330,-7291,-5727,-2678,-950,87,2162,5568,4525,1036,-1171,247,-1665,-952,-2674,-1422,-180,-2947,3287,4389,592,-2020,-1259,-1762,1001,-193,-6302,-3831,-1369,-6581,-5239,-9542,-4156,-3071,-4943,-3071,-1943,-720,-768,-2861,-1422}, {245,-204,92,-290,-619,152,1069,1969,1274,2380,1687,2570,1398,875,1906,2371,2817,2272,-2371,-1014,-582,-668,-727,-1079,-263,-1012,-996,1029,1833,2002,2617,2969,-2631,-619,-3993,-3460,-634,-1066,-1323,-1313,-1651,-2134,-1994,-1645,-3121,-3691,-2011,-1274,-2562,-3056,-1564,876,1999,1760,1446,2948,2292,2287,2849,2105,3613,3713,3109,2712,2840,1862,1375,445,1547,-569,-467,-2662,-1415,1096,1447,-381,944,455,366,-1648,574,2415,577,-54,1045,2009,1737,1499,1610,1934,1424,736,805,936,1635,802,-373,-1046,-1513,-1582,-2703,-2220,-257,921,126,1022,-62,508,335,-104,115,-1758,-1947,-747,175,-91,216,-601,339,2712,2027,1450,2296,1942,4102,1970,1518,2335,1205,-594,871,850,1858,1705}, }; // Frequencies (0 indicates noise) const char F[COUNT] = {0,0,0,0,0,0,0,30,24,25,25,24,26,28,29,32,33,31,31,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,85,0,28,53,0,30,31,33,33,31,0,0,0,0,0,0,0,26,23,0,0,0,0,20,20,20,20,21,21,21,21,21,21,20,31,21,0,0,0,0,0,0,0,0,25,22,22,23,23,26,28,31,32,31,29,26,46,22,21,21,21,0,0,0,0,0,0,0,0,0,0,0,29,28,29,28,29,31,31,29,50,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0}; float Xhat[NW][SIZEOF(G)] = {0}; #define STEP (NW / 2) float xhat[(SIZEOF(G)-1)*STEP+NW] = {0}; void setup(unsigned int seed) { // `lpcDecode` srand(seed); // Random frequency offset for fun. (Doesn't affect formants, just fundamental.) int freq_offset = rand() % 30; float offset = 0; for (int i = 0; i < COUNT; i++) { float ys[ORDER] = {0}; float source[NW] = {0}; if (F[i]) { float step = (float)Fscale/(F[i] + freq_offset); float j; for (j = offset; j < NW/2; j += step) { source[(int)j] = sqrtf(step); } offset = j - NW/2; } else { for (int j = 0; j < NW; j++) { source[j] = noise(); } offset = 0; } for (int j = 0; j < NW; j++) { float x = source[j] * sqrtf((float)G[i]/32767 * Gscale); float y = -x; for (int k = ORDER - 1; k >= 0; k--) { float a = (float)A[k][i]/32768*Ascale; y += a*ys[k]; if (k > 0) ys[k] = ys[k-1]; } ys[0] = y; float frac = (float)j/(NW-1); float w = sqrtf((frac < 0.5 ? frac : 1 - frac) * 2); Xhat[j][i] = w * y; } } // `pressStack` for (int i = 0; i < COUNT; i++) { for (int j = 0; j < NW; j++) { xhat[j + STEP*i] += Xhat[j][i]; } } } float process() { static float i = 0; if (i + 1 >= SIZEOF(xhat)) return 0; int i0 = i; float frac = i - i0; float a = xhat[i0]; float b = xhat[i0 + 1]; i += (float)LPC_SAMPLE_RATE / SAMPLE_RATE; return a + (b - a) * frac; } ```
```c #include
#include "deck.h" card_title("twinkle"); typedef enum { NOTE_ON, NOTE_OFF, } event_type_t; typedef struct { event_type_t type; short time; char pitch; char velocity; } event_t; int ticks_per_beat = 256; int microseconds_per_beat = 631577; event_t events[] = { { .type = NOTE_ON, .pitch = 60, .velocity = 77, .time = 0 }, { .type = NOTE_ON, .pitch = 52, .velocity = 63, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 64, .time = 0 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 60, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 52, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 67, .velocity = 90, .time = 12 }, { .type = NOTE_ON, .pitch = 60, .velocity = 77, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 71, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 67, .velocity = 87, .time = 12 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 69, .velocity = 86, .time = 12 }, { .type = NOTE_ON, .pitch = 60, .velocity = 73, .time = 0 }, { .type = NOTE_ON, .pitch = 41, .velocity = 66, .time = 0 }, { .type = NOTE_OFF, .pitch = 69, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 69, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 41, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 69, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 67, .velocity = 81, .time = 12 }, { .type = NOTE_ON, .pitch = 60, .velocity = 75, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 71, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 487 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_ON, .pitch = 65, .velocity = 80, .time = 25 }, { .type = NOTE_ON, .pitch = 57, .velocity = 70, .time = 0 }, { .type = NOTE_ON, .pitch = 41, .velocity = 68, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 65, .velocity = 85, .time = 12 }, { .type = NOTE_OFF, .pitch = 57, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 41, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 64, .velocity = 82, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 69, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 76, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 64, .velocity = 84, .time = 12 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 62, .velocity = 81, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 71, .time = 0 }, { .type = NOTE_ON, .pitch = 43, .velocity = 69, .time = 0 }, { .type = NOTE_OFF, .pitch = 62, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 62, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 43, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 62, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 60, .velocity = 81, .time = 12 }, { .type = NOTE_ON, .pitch = 52, .velocity = 71, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 75, .time = 0 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 487 }, { .type = NOTE_OFF, .pitch = 52, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_ON, .pitch = 67, .velocity = 94, .time = 25 }, { .type = NOTE_ON, .pitch = 60, .velocity = 80, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 72, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 67, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 65, .velocity = 81, .time = 12 }, { .type = NOTE_ON, .pitch = 57, .velocity = 71, .time = 0 }, { .type = NOTE_ON, .pitch = 41, .velocity = 68, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 65, .velocity = 84, .time = 12 }, { .type = NOTE_OFF, .pitch = 57, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 41, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 64, .velocity = 82, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 70, .time = 0 }, { .type = NOTE_ON, .pitch = 43, .velocity = 73, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 64, .velocity = 84, .time = 12 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 43, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 62, .velocity = 82, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 67, .time = 0 }, { .type = NOTE_ON, .pitch = 43, .velocity = 72, .time = 0 }, { .type = NOTE_OFF, .pitch = 62, .velocity = 0, .time = 487 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 43, .velocity = 0, .time = 0 }, { .type = NOTE_ON, .pitch = 67, .velocity = 90, .time = 25 }, { .type = NOTE_ON, .pitch = 60, .velocity = 76, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 73, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 67, .velocity = 84, .time = 12 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 65, .velocity = 84, .time = 12 }, { .type = NOTE_ON, .pitch = 57, .velocity = 71, .time = 0 }, { .type = NOTE_ON, .pitch = 41, .velocity = 68, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 65, .velocity = 85, .time = 12 }, { .type = NOTE_OFF, .pitch = 57, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 41, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 64, .velocity = 82, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 70, .time = 0 }, { .type = NOTE_ON, .pitch = 43, .velocity = 73, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 64, .velocity = 82, .time = 12 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 43, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 62, .velocity = 83, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 70, .time = 0 }, { .type = NOTE_ON, .pitch = 43, .velocity = 69, .time = 0 }, { .type = NOTE_OFF, .pitch = 62, .velocity = 0, .time = 487 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 43, .velocity = 0, .time = 0 }, { .type = NOTE_ON, .pitch = 60, .velocity = 82, .time = 25 }, { .type = NOTE_ON, .pitch = 52, .velocity = 69, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 73, .time = 0 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 60, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 52, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 67, .velocity = 90, .time = 12 }, { .type = NOTE_ON, .pitch = 60, .velocity = 80, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 72, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 67, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 69, .velocity = 87, .time = 12 }, { .type = NOTE_ON, .pitch = 60, .velocity = 69, .time = 0 }, { .type = NOTE_ON, .pitch = 41, .velocity = 68, .time = 0 }, { .type = NOTE_OFF, .pitch = 69, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 69, .velocity = 81, .time = 12 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 41, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 69, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 67, .velocity = 79, .time = 12 }, { .type = NOTE_ON, .pitch = 60, .velocity = 72, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 73, .time = 0 }, { .type = NOTE_OFF, .pitch = 67, .velocity = 0, .time = 487 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_ON, .pitch = 65, .velocity = 81, .time = 25 }, { .type = NOTE_ON, .pitch = 57, .velocity = 69, .time = 0 }, { .type = NOTE_ON, .pitch = 41, .velocity = 68, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 65, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 57, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 41, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 65, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 64, .velocity = 81, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 70, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 75, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 64, .velocity = 84, .time = 12 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 64, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 62, .velocity = 82, .time = 12 }, { .type = NOTE_ON, .pitch = 55, .velocity = 71, .time = 0 }, { .type = NOTE_ON, .pitch = 43, .velocity = 69, .time = 0 }, { .type = NOTE_OFF, .pitch = 62, .velocity = 0, .time = 244 }, { .type = NOTE_ON, .pitch = 62, .velocity = 83, .time = 12 }, { .type = NOTE_OFF, .pitch = 55, .velocity = 0, .time = 231 }, { .type = NOTE_OFF, .pitch = 43, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 62, .velocity = 0, .time = 13 }, { .type = NOTE_ON, .pitch = 60, .velocity = 80, .time = 12 }, { .type = NOTE_ON, .pitch = 52, .velocity = 70, .time = 0 }, { .type = NOTE_ON, .pitch = 48, .velocity = 74, .time = 0 }, { .type = NOTE_OFF, .pitch = 60, .velocity = 0, .time = 487 }, { .type = NOTE_OFF, .pitch = 52, .velocity = 0, .time = 0 }, { .type = NOTE_OFF, .pitch = 48, .velocity = 0, .time = 0 }, }; typedef struct { float freq; float phase; float time; float env; float amp; bool held; } voice_t; voice_t voices[128] = {0}; const float attack_time = 0.1; const float decay_time = 0.2; const float sustain_level = 0.5; const float release_time = 0.5; #define SAW(phase) ((phase) * 2 - 1) float generate_envelope(int i) { if (!voices[i].held && voices[i].env == 0) { return 0; } voices[i].time += dt; float t = voices[i].time; if (t < attack_time) { return t / attack_time; } t -= attack_time; if (t < decay_time) { return 1 - (t / decay_time) * (1 - sustain_level); } t -= decay_time; // debug("t: %f\n", t); if (voices[i].held) { // debug("sustain\n"); return sustain_level; } else if (t < release_time) { // debug("%f\n", sustain_level * (1 - t / release_time)); return sustain_level * (1 - t / release_time); } else { // debug("done\n"); return 0; } } float synthesize_waveform(int i) { return saw(&voices[i].phase, voices[i].freq); } float synthesize_voices() { float out = 0; for (int i = 0; i < SIZEOF(voices); i++) { voices[i].env = generate_envelope(i); if (!voices[i].env) continue; out += voices[i].env * voices[i].amp * synthesize_waveform(i) / 8; } return out; } float velocity_to_amplitude(char velocity) { // Reference: https://www.cs.cmu.edu/~rbd/papers/velocity-icmc2006.pdf const float R = 30.0; // dynamic range in decibels const float r = powf(10.0, R / 20.0); const float b = 127.0 / (126.0 * sqrt(r)) - 1.0 / 126; const float m = (1.0 - b) / 127.0; return powf(m * velocity + b, 2.0); } float process() { static int i; static float next_event_time, t = 0; gen_begin; for (i = 0; i < SIZEOF(events); i++) { next_event_time = (float)events[i].time / ticks_per_beat * microseconds_per_beat / 1e6; for (; t < next_event_time; t += dt) { yield(synthesize_voices()); } t -= next_event_time; debug("%d %d %d\n", i, events[i].type, events[i].pitch); int p = events[i].pitch; if (events[i].type == NOTE_ON) { voices[p].freq = m2f(p); // May be a retrigger; jump to appropriate point in ATTACK segment. voices[p].time = attack_time * voices[p].env; voices[p].amp = velocity_to_amplitude(events[i].velocity); voices[p].held = true; } else if (events[i].type == NOTE_OFF) { // Jump to appropriate point in DECAY or RELEASE segment, depending on envelope amplitude. if (voices[p].env > sustain_level) { voices[p].time = attack_time + decay_time * (1 - voices[p].env) / (1 - sustain_level); // debug("a: %f\n", voices[p].time); } else { voices[p].time = attack_time + decay_time + (1 - voices[p].env / sustain_level) * release_time; // debug("b: %f\n", voices[p].time); } voices[p].held = false; } } for (;;) { yield(synthesize_voices()); } gen_end(0); } ```
--- # Discussion - Code as Compression - Musical Processes/Computational Processes - Pointers and Values - Content repositories; control and access - "Pass-by-reference" vs. "Pass-by-value" - Musical mediums - Tangibility; the value of physical "cards" --- # Future work - More approachable interfaces for creation - More methods for manipulation: "punch cards" - Visualizuation, execution tracing - ScoreCard as UGen? - Visually interesting codes? --- # Thank you! ## [github.com/ijc8/scorecard](https://github.com/ijc8/scorecard) ### [ijc8.me](https://ijc8.me) — ijc@ijc8.me Note: TODO Here's my card