Work · 05 of 9

KisanMind

Five-node LangGraph advisor for farmers with Hindi and Marathi voice, built in 24 hours

Role
AI service (agents, orchestration, voice, tests) and frontend wiring
Team
Ved Kumar Singh, Nimit Jain, Parth Kshirsagar
Period
Apr 2026
Status
hackathon
Stack
  • Python
  • FastAPI
  • LangGraph
  • LangChain
  • Gemini 2.0 Flash
  • Sarvam AI
  • Groq Whisper
  • Next.js 15
  • MongoDB
  • Vercel
  • Render
altahackathon.vercel.app
KisanMind, the advisor interface, captured 2026-09-15
KisanMind, the advisor interface. Captured from the live site on 2026-09-15.

Top 2 of about 40 teams, HackWarts by Alta School of Technology

Problem

A smallholder farmer deciding what to plant needs soil, weather, mandi prices and scheme eligibility at once, and that information lives in four places and two languages. Most of the people who need it do not type; they talk. HackWarts gave us 24 hours. I took the AI service end to end: the agents, the graph, the voice chain, the tests, and the frontend wiring that put agent output on the dashboard. Ved and Nimit took the Next.js pages and auth, Parth the Node backend and MongoDB.

From the repository

113Commits45 mine, 40%
20,189Lines of code154 source files
10Test filesin the tree
Counted from AltaHack + kisanmind-alerts on 2026-09-15, commits 2026-04-10 to 2026-04-15. Source lines only (no vendored, generated or built files). Four contributors; my 45 commits are the AI service and the frontend integration.

The AI service is FastAPI with LangGraph: five nodes over one shared TypedDict state, seven endpoints (/orchestrate, /crop/recommend, /market/analyze, /financial/analyze, /chat/follow-up, /chat/voice, /health), four pluggable LLM providers behind one llm_service, and 152 test functions in nine files, sixty of them for voice and language handling.

The pipeline

Farmer profiletext or voiceWeathergeocode, Open-MeteoCroprules → score → LLMre-rankMarketmandi, MSP, windowInsightsrisk, action planFinancialschemes, credit, ROI
A LangGraph state graph. Each node reads the shared state and writes one key.

State, not messages. The graph carries one SharedState: farmer_profile, then weather_data, crop_output, market_output, financial_output, insights_output, final_output. Each node reads what it needs and writes exactly one key, so a failing node leaves a typed hole rather than a corrupted conversation, and the tests can assert on the state after any single node.

The crop node is deliberately not an LLM call. It loads a catalogue of 25 crops, eliminates by four hard rules (water, season, soil, budget) and records a reason for every crop it drops, scores the survivors on five weighted factors, and only then asks Gemini to re-rank the top three with reasoning. The reply is parsed against a schema; a bad reply is retried once and then replaced by the rule-based ranking, so the endpoint always answers. Rejection reasons are attached after the model call, so the model cannot invent them. That is what makes the recommendation explainable, and what makes it testable.

Market and financial. The market node pulls mandi prices, compares with MSP, computes a trend and a selling window, and picks a buyer channel before the LLM writes the summary. The financial node is rule-matched eligibility for PM-KISAN, PMFBY, KCC and PM-Kusum with a rough ROI and credit need. The insights node reads all of it, looks for contradictions between agents (a crop the market node says will not sell), and writes the action plan with a confidence.

Chat. Follow-up questions go through a keyword router in English and Hindi (crop, financial, market, “what if” profile updates, insights, general) and a session store that keeps the crop ranking, so “what about the second one” works.

Try it

The crop node’s first two steps, elimination and scoring, run here with the same rules, weights and catalogue as the repository. Change the farm and watch which crops get rejected and why, and which three would reach the model.

Crop agent, rules and scoringcrop_rules.py over the 25-crop catalogue, before the LLM sees anything
Viable, scored14 of 25
Rejected by rule11
  • Wheat Not suitable for Kharif season
  • Rice Incompatible with Black Soil
  • Chickpea Not suitable for Kharif season
  • Mustard Not suitable for Kharif season; Incompatible with Black Soil
  • Potato Not suitable for Kharif season; Incompatible with Black Soil
  • Barley Not suitable for Kharif season; Incompatible with Black Soil
  • Lentil Not suitable for Kharif season; Incompatible with Black Soil
  • Turmeric Incompatible with Black Soil
  • Jute Incompatible with Black Soil
  • Peas Not suitable for Kharif season; Incompatible with Black Soil
  • Watermelon Not suitable for Kharif season

Rules and weights are exactly those in the repository: water, season, soil and budget eliminate; soil .30, water .25, season .20, weather .15, budget .10 rank. The top three go to Gemini with the farmer profile and a weather summary, and the reply is parsed against a schema, retried once, and replaced by this rule-based ranking if it still fails. Rejection reasons are attached after the LLM call, so the model can never invent one. 152 tests in the repository cover these paths.

Voice, in 24 hours

Browser audio goes to /chat/voice, is transcribed by Sarvam Saaras (22 Indian languages) with Groq Whisper as fallback, runs through the same chat handler as text, and comes back as Sarvam Bulbul speech with ElevenLabs as fallback, as base64 audio. Every one of those hops broke at least once on the second day.

BeforeAfter
Speech to textBrowser webm/opus rejected by the STT APIFile tuple with the right MIME, mp4 preferred, codec stripped; Sarvam Saaras first, Groq Whisper fallback
Hindi transcriptsCame back in Urdu scriptUrdu to Devanagari correction and per-session response language
Text to speechTruncated long answers, no fallbackSarvam Bulbul with a length cap, ElevenLabs fallback, browser TTS as the last resort
FrontendDuplicate messages, audio persisted into local storageOne voice hook, safe playback, audio stripped before persisting
Config.env lost on uvicorn reloadAbsolute-path load_dotenv

Fifteen tests were added for audio validation alone: MIME handling, file extension safety, TTS truncation, and boolean parsing of form fields.

Timeline

  1. Folder structure and instructions
  2. AI service scaffolded with the crop pipeline then financial agent, session store, chat router, voice service, language detection, insights agent, xAI as the fourth provider.
  3. Dashboard wired to live agent cards, chat and voice VoiceRecorder with a real-time waveform; Insights page.
  4. 27 tests added with the README
  5. The Hindi voice day webm/opus rejection, Urdu script transcripts, response language, TTS fallback, .env on reload; 15 audio tests.
  6. Structured rejection reasons in CropOutput plus tests for rejection reasons, routing and fallback.
  7. Prompts tightened for advisory quality; architecture doc with diagrams
My commits from git log, over the hackathon's 48 hours.

What is missing

Evaluation. The pipeline has tests but no measured quality: no agreement rate against an agronomy reference, no word-error rate on Hindi speech. The schemes are rule-matched from a static list, not live. Until a 30-profile evaluation set exists and its results are published here, this page does not carry a measured mark. That is the next piece of work on this project, and it is the same discipline I apply to the evaluation lab on this site.

Learned: Voice for people who don't type: every layer needs a fallback.

keysKeyboard
j / k
next / previous row
Enter
open the focused row
⌘K or /
search, or ask this site
g then h w a c
go home, work, about, contact
t
toggle light and dark
Esc
close the palette or this map
?
this map