In a noteworthy stride towards linguistic innovation, Indian AI startup Sarvam AI has released OpenHathi LLM, marking a significant leap in the realm of Hindi language models. Just a week after securing an impressive $41 million in Series A funding, the company introduced OpenHathi-Hi-v0.1, the inaugural release in the OpenHathi series.
Genesis of OpenHathi
Sarvam AI’s latest creation is rooted in Meta AI’s Llama2-7B architecture, specifically tailored for the nuances of Hindi, India’s most widely spoken language. Positioned as the first Hindi Large Language Model (LLM) in the OpenHathi series, it promises performance on par with GPT-3.5 for Indic languages. The model’s foundations lie in a budget-friendly platform, extending the capabilities of Llama2-7B.
Unveiling the Training Process
OpenHathi-Hi-v0.1 undergoes a meticulous two-phase training process. Initially, the focus is on embedding alignment, aligning randomly initialized Hindi embeddings. Subsequently, the model engages in bilingual language modeling, mastering cross-lingual attention across tokens. The result is a robust performance across various Hindi tasks, showcasing prowess in both native and Romanized scripts.
Collaborations and Academic Contributions
Developed in collaboration with academic partners at AI4Bharat, Sarvam AI’s OpenHathi-Hi-v0.1 benefits from language resources and benchmarks provided by these partners. This collaborative effort extends beyond linguistic boundaries, as seen with KissanAI’s recent announcement of Dhenu 1.0, a groundbreaking Agriculture Large Language Model. This bilingual marvel caters directly to farmers’ linguistic needs in English, Hindi, and Hinglish.
Road Ahead for Sarvam AI
Pratyush Kumar and Vivek Raghavan, the co-founders of Sarvam AI, propelled the startup into existence in July 2023. Bolstered by substantial Series A funding led by Lightspeed Ventures, the duo envisions addressing India’s unique needs through Generative AI integration for diverse Indian languages. Their focus extends to fostering collaborations with enterprises for domain-specific AI model development using data as the backbone.
Our Say
In a landscape where linguistic diversity is paramount, Sarvam AI’s OpenHathi-Hi-v0.1 emerges as a promising addition, embodying a commitment to innovation in Indian language AI. The collaborative spirit with academic partners and a clear roadmap for the future positions Sarvam AI as a torchbearer in the Generative AI domain. As the model beckons developers to explore its potential, we anticipate a ripple effect of creativity and specialized models that will further enrich India’s AI landscape.
Adhering to the principles of transparency and collaboration, Sarvam AI’s journey from research and digital infrastructure development to pioneering language models demonstrates a conscientious approach to addressing the unique linguistic tapestry of India.
