Project

Harmonova

Production-oriented intelligent music-system architecture

Experimental learning project

Role: Creator, software architect, deterministic domain modeling, Spring AI orchestration, MCP boundary design

Harmonova explores how deterministic domain models, retrieval, AI orchestration, and specialized analysis services can work together in a production-oriented intelligent system.

Music theory stays inside a pure Java core, while AI components handle interpretation, planning, and recommendations around verified facts instead of becoming the source of musical truth.

Back to the homepage projects overview

Problem

LLMs are helpful for conversational composition workflows, but they are unreliable for exact scale spelling, chord progressions, or deterministic beat-generation rules on their own.

At the same time, audio, DSP, and ML workloads can quickly pollute an otherwise clean domain core unless their boundaries are enforced deliberately.

Approach

Harmonova keeps theory and beat logic inside a deterministic domain core, then exposes that core through MCP tools and service APIs where orchestration is useful.

A Spring Boot layer coordinates tutor and beat-maker flows with retrieval, provider routing, and tool use, while a separate Python service handles audio-analysis workloads behind a narrow boundary.

  • Deterministic work stays in the core; LLMs are reserved for natural language and orchestration.
  • Dependencies point inward: web and MCP depend on core, but core depends on neither.
  • Deterministic endpoints can bypass MCP and LLM layers when conversational orchestration is unnecessary.
  • Audio bytes, DSP, and ML remain isolated in Python instead of leaking into the Java core.

Architecture

The system is organized around four codebases: harmonova-core, harmonova-mcp-server, harmonova, and harmonova-audio-analysis-service.

The web application and MCP server depend on harmonova-core, while the Python audio-analysis service stays outside that dependency graph and is reached through a thin HTTP boundary.

Agent workflows are designed around a PLAN -> FACTS -> RAG -> COMPOSE -> VERIFY -> EMIT pipeline so model output can be checked against deterministic tools and retrieved context.

In progress

harmonova-core

The deterministic music-theory and beat-generation core is the center of the system and is kept free of Spring, AI, and audio dependencies.

In progress

harmonova-mcp-server

The MCP server wraps deterministic core operations so model-facing orchestration can use tools without polluting the domain layer.

In progress

harmonova

The Spring Boot application carries REST endpoints, agents, retrieval, provider routing, and the user-facing orchestration layer.

In progress

harmonova-audio-analysis-service

The Python and FastAPI audio-analysis service contains the DSP and ML workload that intentionally sits outside the pure domain core.

Key Decisions

Deterministic core versus probabilistic orchestration

Music theory, beat composition parameters, and fact verification remain in the pure Java core so the AI layer cannot invent exact musical facts.

MCP boundary instead of direct tool coupling

Tool definitions stay in the MCP server, which makes the domain core reusable without dragging web or AI concerns into it.

RAG grounding for conversational flows

The tutor and beat-maker flows use retrieved material to ground prompts rather than relying on unconstrained model generation.

Python ML isolation

Keeping audio analysis outside the Java core reduces domain pollution at the cost of another service boundary.

Tradeoffs

More moving parts than a single app

Polyrepo ownership, MCP transport, vector storage, and the Python service make the system more modular, but also more operationally involved.

Local-first options still require infrastructure

Ollama, pgvector, reranking, and observability can stay self-hosted, but that local-first posture still brings setup overhead that a simpler demo would avoid.

Deterministic boundaries limit shortcutting

The design deliberately prevents the LLM from becoming the source of musical truth, which improves correctness but can slow down rapid prototyping.

In progress

Deterministic music theory in core

In progress

Active development centers on a pure Java core for scales, chord progressions, melody analysis, seeded beat composition, and MIDI-oriented domain operations.

Agent orchestration over deterministic facts

In progress

The tutor and beat-maker flows are being built so Spring AI agents plan and explain around deterministic tools instead of inventing theory facts.

Retrieval and tool integration

In progress

RAG, local retrieval components, and MCP tools provide the grounding layer around the core domain model.

Isolated audio analysis service

In progress

Audio-analysis and ML workloads are being kept in a separate Python service so the core and orchestration layers stay focused.

Planned

Stronger audio classification path

Planned

The roadmap includes moving beyond heuristic genre and mood analysis toward a stronger classification path.

Technology Stack

  • Java
  • Spring Boot
  • Spring AI
  • MCP
  • Ollama
  • PostgreSQL
  • pgvector
  • Infinity reranker
  • Python
  • FastAPI
  • librosa
  • PyTorch
  • MERT
  • OpenTelemetry
  • Langfuse
  • Prometheus
  • Loki
  • Grafana
  • Docker Compose

Local-first and boundary choices

  • Harmonova favors local-first AI components where that supports control and experimentation, including local model, embedding, and reranking options.
  • The architecture keeps audio bytes, DSP, and ML behind a single HTTP boundary instead of letting those concerns spread through the rest of the Java system.
  • Provider routing stays explicit so deterministic domain logic can remain stable even as AI components evolve.

Current Status

Harmonova is an active experimental system. This page distinguishes the deterministic core, the services being developed around it, and the roadmap still ahead.