Last night I was guest speaker at the Boston .NET Architecture community group. I learned they are now 21-year-olds. That’s a long track record! The audience had some insightful questions, which I always appreciate.
My talk focused on the perspective of the application architect – and not the data scientist, for example – in how the process works and what are some areas I would need to dig into.
Here’s the alternative talk description I offered a few days ago:
Interested in understanding how LLMs are created and how they work internally, including all the in-depth data science and machine learning techniques? If so, then this is not that talk. Rather, this talk steps back to treat the LLM as a black box. And then steps further back to treat the LLM as a part of a cohesive system offered over the internet through an API. It is from that perspective that we begin our exploration.
How exactly does an application make use of LLM services? Is this thing secure? Is it private? Am I operating according to Responsible AI principles? (Oh, and what are Responsible AI principles?) Is it accurate? Is it portable? And of course, when does it stop being a Chatbot and start being an Agent?
These are some of the key types of application architecture considerations we will discuss as we start with “the humble chatbot demo” then turn it into an Agent and then see what it would take to put that into production.
The deck is here:
The recording is here: https://youtu.be/UJutO4eFLZg
