Controlling AG Grid with Natural Language using Hashbrown

The AG Grid team dropped a new release that includes a new AIToolkitModule that provides developers an interface for reading and writing the grid state using an agent.

Of course, when we saw this, we were excited. In fact, a few months prior, Mike and I met with some of the folks at AG Grid to show them how easy it was to use an agent in the browser using Hashbrown to enable the user to modify the state of the grid using natural language. We were excited to learn that they were already working on developing an API to support this approach.

This new enterprise module ships a new getStructuredSchema() method that returns JSON schema. Combined with the grid API getState() and setState() methods, we can expose the grid state schema to the model, along with tools for getting and setting the grid state.

Improving Gemini’s Structured Response Data

Google announced updates to expanded support for JSON Schema when specifying the response format. Previously, gemini models were using a very limited subset of OpenAPI schema. The update adds support for:

  • anyOf unions

  • $ref for recursive schemas

  • minimum and maximum for numeric constraints (which was only recently supported by OpenAI models)

  • additionalProperties can be null

  • prefixItems for tuples

What does this mean for building browser-based agents?

The move to JSON schema and expanding the support surface is a win for developers.

But, I still run into issues using JSON schema with gemini models. Worse, the Gemini API often returns a 400 with an unhelpful error message. Ugh.

The reality is that today’s models do not support 100% of the API surface of JSON schema. For example, the multiples option allows you specify that number values must be a multiple of a given number. Cool. The problem (for now) is that if I specify this valid JSON schema, the model may puke (eh hem, looking at you my friend) or just ignore it. ¯\_(ツ)_/¯

So, what can we do?

Well, if you’re crazy (like Mike) you pitch the Hashbrown team on the fact that we need to build a schema language that is LLM-optimized.

What does that mean?

Skillet, hashbrown’s schema language, is highly descriptive and intentionally limited to schema that is broadly aligned with the schema supported by the model APIs.

Can AI run a (very) small business?

The Anthropic team, along with their partners Andon Labs, wrote about an experiment to test if Claude could run a business.

The idea was pretty simple. Build an agent to run a small shop. They provided the agent with tools for managing stock, sending emails to vendors, and a slack integration for listening to customers (Anthropic employees).

The whole story is pretty fun to read - and a bit disturbing - but nothing we haven’t seen in this age of AI hallucinations and gaslighting.

Bites

Keep Reading

No posts found