Back to home

AI and music Prototype 2026

Neuro-Candombe

A Transformer that plays chico, repique and piano drums.

Where it came from

Montevideo's candombe has very precise rules —the chico barely changes, the piano marks the neighborhood, the repique improvises— and at the same time a groove that doesn't fit on a grid. I wanted to know whether a model could learn both things together: the rule and the offset.

The problem

Rhythm generators quantize everything to the grid. In candombe that erases what matters most: the chico's microtiming, the articulation of each stroke and the shape of the llamada. A generic model also doesn't know that some things are non-negotiable, like the chico pattern.

The solution

A bidirectional masked Transformer over a sixteenth-note grid for the three drums. Each cell stores intensity, microtiming offset and articulation. The playing rules aren't applied afterward: they go in as fixed cells and allowed-class masks during decoding. A FastAPI API generates, fills in one drum over others held fixed, or redoes bars, and a web frontend plays the llamada and exports WAV and MIDI.

What's solved

  • Per-cell representation: intensity (silence + 4 levels), microtiming in 13 bins and articulation per drum (hand, stick, wood, open, muffled).
  • Iterative confidence-based decoding, with the playing rules as hard constraints rather than a post-hoc filter.
  • Three core styles —Cuareim, Ansina and Gaboto— loaded from published transcriptions, with the source next to each pattern.
  • Microtiming taken from published measurements (Jure and Rocamora, 2022) and llamada form with "subir la llamada".
  • POST /regenerate: redoes specific bars or drums and keeps the rest fixed; the seed makes every result reproducible.
  • Objective evaluation with copy_rate to detect whether the model memorizes bars from the corpus, plus a blind listening test flow.
  • Frontend "parade of llamadas": ensemble size, panning per drummer, lookahead scheduling and WAV/MIDI export.

Evidence

  • 31 pytest tests covering representation, generation, training, audio analysis, styles and API.
  • Importer for the UdelaR candombe dataset (Zenodo 6533068: 35 performances, ~2 h annotated by an expert).
  • Analysis of my own recordings by stems or from a mix, with sample-level onset refinement.
  • Separate deploys: static frontend on Netlify and API on Render, with a fallback to a rule-based engine if there is no checkpoint.

The hard parts

  • The first version didn't learn: the target was visible in the input. Moving to a masked Transformer, which only scores hidden cells, was the change that unblocked it.
  • Candombe microtiming isn't noise. It had to be represented as its own class per cell instead of quantizing it and "humanizing" at the end.
  • Separating drums from a mixed recording is a band-based heuristic: soft ghost notes get lost. With per-drum stems the problem goes away.

What I learned

  • Domain rules work better as decoding constraints than as after-the-fact correction: the model generates within the valid space.
  • Measuring the copy rate keeps you honest: a model that sounds good may be repeating the corpus.
  • Citing the source of every pattern makes the system something you can discuss with musicians, not just engineers.

Next steps

  • Train on the real dataset and my own multitrack recordings: the included checkpoint has only seen the synthetic corpus.
  • Replace the synthesized strokes with a bank of recorded samples.