Developers traditionally work with computers using a language created for the purpose, which they have had to learn. The GenAI revolution lets developers work with computers using natural language instead.
However, natural human language isn’t as easy for computers to understand. The ambiguities and inconsistencies of natural language make it impossible for computers to respond as precisely as they do for programs written using a computer language.
Nonetheless, the recent AI Summit of New York conference was buzzing with natural human language discussions about exactly what GenAI is good for, how developers should use it, what tools to use, how to create revenue from it and how to use it for good (NY Governor Hochul).
About 4,000 visitors attended the conference, which featured about 400 speakers and 100 exhibitors, including startups and major corporations.
Challenge of GenAI
The errors, hallucinations, bias issues and poor quality of the results from GenAI programs make it difficult to figure out the best applications of GenAI — and especially what organizations will pay for you to develop with it.
It’s a bit like a dancing bear: You’re amazed the bear can dance in the first place, but it’s a long way from that to the $300 seats for “The Nutcracker” at New York City Ballet.
At this point, at least, it seems like developers and analysts must be in the loop to review and approve GenAI output. Talk of autonomous agents seems to be mostly talk, at least for now.
Breadth of GenAI Use Cases
Use cases highlighted at the conference focused on potential productivity gains for developers, but also on a wide range of applications for which developers can use GenAI.
Developers use AI code assistants to create GenAI applications for use cases such as life sciences, government, healthcare, banking, travel, utilities, smart cities and so on.
Representatives from GenAI-consuming organizations, GenAI vendors and governments talked about their use cases, and proposed solutions and efforts on behalf of voters.
Lemony AI
One of the commonly accepted use cases is developing chat applications for interacting with complex documents, such as HR policies, bank account onboarding or legal rulings. Conference exhibitor Lemony AI sells AI software designed for this use case, packaged within specially designed hardware modules.
They offer the hardware so that external connections (such as to the cloud) are not required for their GenAI applications, which helps meet data regulations, ensures data privacy and enables additional training data usage. Organizations own their own AI models.
“We sell the software service and you get the hardware for free,” Lemony AI CEO and co-founder Sasche Buehler told The New Stack.
Customers subscribe to one of the Lemony AI plans and receive one or more hardware modules, depending on the plan, with generative AI software preinstalled. Subscribers receive regular software updates.
Subscriptions are $499 a month for a single box, Buehrle explained. “Customers typically start with a single box and after two to three months upgrade to a four-node subscription for $1299 a month,” he said. Clusters of nodes share resources.
“We don’t want the box back,” added Buehrle. The boxes include six terabytes of flash storage and four AI accelerator processors of their own design.
Users upload their documents and Lemony AI’s software transforms them. “We only store the vector data on the box,” Buehrle said. The users then use the Lemony AI software to chat with the document data.
Lemony AI software includes an LLM and agents for document related workflows, such as executing a contract and sharing confidential information with those allowed to view it.
Answer Rocket
Another use case for GenAI that appears to be gaining acceptance is using a chat interface for interaction with analytics data.
Mike Finley, CTO and co-founder of conference exhibitor AnswerRocket, told The New Stack that customers want to ask questions of their data, such as “Why is my market share down?”
“Suddenly we could put AI in the middle” of the data pipeline, Finley said, creating “AI for BI.”
The result is “conversations as natural as ChatGPT but 100% based on facts,” Finley added, connecting GenAI to corporate data.
Using the traditional BI approach takes too long to get answers, he explained. But using AI introduces the risk of incorrect replies.
“Models are trained to please people and to be polite. If you underspecify a prompt, the LLM has to make up an answer,” Finley said. AnswerRocket therefore uses a multi-prong approach to solve the hallucination problem.
Making AI Safe for Humans
Whatever the use cases are for GenAI, developers and users are justifiably concerned about its risks.
Greg Whalen, CTO of conference exhibitor Prove AI, told The New Stack that it’s “all about de-risking AI.” That seems to be the thinking we can all rally around, he added.
“Everyone wants to develop as quickly as possible. While we expect the software stack to evolve, compliance and risk management will require continuous attention. What is there is not sufficient,” Whalen said.
Prove AI leverages the Hedera blockchain for certifiable and tamper-proof auditing for an organization’s compliance and risk policies related to developing and running AI models, and makes the results visible to third parties as needed.
Prove AI is a SaaS-based product with connectors for integrating with and governing an organization’s data sets and LLMs, and provides a set of preconfigured compliance modules for ISO and SOC standards, such as ISO 42001, for example. Compliance models are customizable and auditable.
NY Gov. Kathy Hochul and Empire AI
Meanwhile, New York Gov. Kathy Hochul gave an unannounced keynote on the event’s second day. She highlighted the importance of AI as a way to improve the functions of government, and to drive business for city and state tech companies.
New York’s Empire AI Consortium — consisting of private companies, government and academic institutions — recently installed a $16.5 million supercomputer powered by 24 Nvidia DGX systems, which contained 192 Tensor Core GPUs.
Gov. Hochul said that this supercomputer, installed at the University of Albany, is “more powerful than anything that has ever been in public hands before.” She said the intention is to “set a model for how AI can be used for public good.”
Hochul mentioned lowering the price of groceries, farming efficiency, health care outcomes and a “fairer, more efficient society for our children to inherit” as goals for the project.
The consortium said they have already received more than 100 research proposals for the supercomputer.
The post Dancing Bears to Ballet: GenAI’s Path to Enterprise Value appeared first on The New Stack.