Welcome to another episode of Interact’s Grow Podcast, where we bring you the latest updates and insights from our team. In this episode, we take you behind the scenes of our newest product, Interact AI!
We discuss the year-long journey of developing Interact AI, and our special guest from the engineering team, Evan, shares the challenges, triumphs, and the iterative process behind building it. We also delve into how we’ve incorporated feedback from our team and customers to refine the product. Tune in to learn more about the exciting features of Interact AI!
Ready to put our AI-powered quiz maker to the test? Get started here!
Hi guys, and welcome back to Interact’s Grow podcast. So great to be with you as always. I’m your host, Jessmyn Solana. We have Jackie, Damaris, and Jesy here with us today. And a very special guest from the engineering team, Evan. Welcome back. Woo woo!
You guys might remember Evan from a previous episode that we did, I think we talked mostly about company culture, which was a really good episode. So go look for that. Check it out. If you haven’t yet, um, and it’s been a while since we had someone from the engineering team come on and we’re nearing the end of the year, nearing 2024.
So we thought it would be a good idea to kind of dig in to what engineering has been doing and sort of. Like this past year working on interact AI. So Evan, I kind of want to leave the floor open to you. And then ladies, if you have any questions, feel free to jump in and then just take it away. Yeah, well, thank you for the introduction and thanks for letting me join.
I, I got very festive since we’re in the love the Christmas tree ready to go. So, yeah, just some updates from, like, the engineering side. What have we been working on the past couple of months? thEre’s been 2 tracks of work on the engineering side. Yeah, 1. Exciting and one more left less exciting. In the background, we’re always doing like tech network and technical debt reduction, trying to make our code base cleaner, trying to unify our APIs.
Make it easier to debug things, make things faster, all the stuff that isn’t like a brand new feature, but makes the systems run smoother. So that’s been going on in the background. We’re having lots of discussions around. How to kind of consolidate all of our code into one single place. But on the more exciting, sexy we’ve been doing a lot of stuff around AI and AI quiz generation.
So a big feature that we’ve been working towards is the ability to automatically create an interact quiz with the help.
Of such that you can put in your quiz title idea, or just your general idea for a quiz, some outcomes if you have them, and then we’ll take that information. Combined with content from your website or social media presence to create a quiz. Which highlights your company. In an interactive way. So doing that, you know, it’s a pretty big task going to that glorious end goal of having it be this automated system.
And we’ve been slowly taking steps toward that. I think a brief history, which everybody on this podcast is aware of. We started from very humble beginnings of this idea of let’s generate a quiz with AI. I think it really started with. A set of prompts or recipes that Josh made and the team here would use within like the chat or open a playground.
Combining those preset prompts with. Information we got from customers that process as it sounds works, but is really, really manual. It’s error prone. Sometimes we don’t have consistency and, like, the way that the prompts are being run. And it just, yeah, it takes a lot of time from the team. So while it was a great way to get people on boarded.
To interact and, like, show them the power of quizzes. It wasn’t really scalable with our team. We’ve got a million requests per day, 200, 000 manual quiz generations per year. That doesn’t really sound like we’ve built a lot of quizzes. That’s for sure. In the past few months, a lot of quizzes. So we’ve been working towards like automating more and more pieces of that flow and one tool that wasn’t released externally, but was like an.
An internal admin tool was a automation of those prompts, um, such that admins here could just click a button and gather information from a user’s website and take their quiz request and then automatically run it through those prompts as opposed to using the playground. So that was a big step towards automating.
But. It’s still had its issues, right? Sometimes the prompts would break. Sometimes we didn’t have enough data. We couldn’t get data from a user’s website. So there was a ton of edge cases that still exist today. Shout out to you, Evan, for doing that, because that made our lives so much easier. Yeah, my part doing my part.
It was amazing. And with that said, I got to shadow, like, when I was making this shout out back to you, Damaris for letting me just, you know, be in the wall and see you use the playgrounds deal with, like, these edge cases, talk me through, like, how you figure them out or like your workarounds. Because we incorporated a lot of your feedback and the team’s feedback into making this dashboard synergy, right?
Yeah, for sure. Like, the last step here and kind of the, you know, towards that, like, glorious in feature is the full automation. Of taking someone’s quiz idea and then actually creating a quiz and putting it in there into their account. And very excitingly, we’re getting very close to having this feature and production ready.
Such that you can fill out the questions, hit, submit, create a new account or link to your account. And in, like, 5 minutes or less, you should have a brand spanking new quiz in your account. So it builds off the admin dashboard takes that logic and, you know, extends it adds even more, you know, functionality to it and add some, like, design elements as well to make the quiz look good.
And try to match your brand, um, and takes a lot more work off of the team’s plate, hopefully. So, yeah, it’s been a very, you know, iterative process on the inch team. We are constantly talking with our team internally and shout out to Josh for constantly getting feedback from in customers and you all to saying like, Hey, do you like your quiz?
This is what part. We know what parts were good, what parts were bad, and we incorporate that feedback. Into the prompts that we use and into, like, the quiz construction process to make it all better. Yeah, shout out to Josh. He’s a, he’s the goat for sure with this quiz creation stuff, man. I don’t know how he does it, but I mean, I guess he’s been doing this for 10 plus years, but holy moly.
Shout out, boss. Yeah, and shout out to the whole engineering team. Like, now, when you’re just going through all this, when I look back, like, wow, like, our whole team really, like, all contributed and worked on this together more than ever. Like, I don’t think I’ve worked more closely with the engineering team or even Jackie or Numeris or anyone, and it came out so good.
Really goes to show that working together like this came out. Awesome. Yeah, to give perspective. We started the project in January, probably actually, like, December, November, December if you want to go back to probably when Josh was like, this would be a great idea. So late last year, and this has really been, like, a year long project very close.
And so that leads to my next question. Actually, Evan. The, you know, you have to build this pure automated process while, you know, we’re actually creating quizzes using the AI. What are the steps that you think, like, it doesn’t happen overnight. So what are the steps that you guys had to go through to make sure that.
This product, when it does go live and it launches is something that anybody can use. Yeah, that’s a great question. Do we have, like, 2 or 3 days to discuss? Yeah, like, you’re right. Like, there’s a lot of considerations that go into Into building something that’s automated extensible can handle, like, a large variety of use cases.
I’d say 1 kind of foundational thing that we focused on was that idea of. I’ll call it prompt construction, not trying to make it sound more complex than it is, but, at the very foundation of this is the idea of, like, having some preset prompts or instructions, which output a quiz and combine those with data that we get from some user, whether that be something they submit directly to us, or it comes through their website or social media profile or some other data source.
So something we spent a lot of time thinking about and working on with making like this prompt construction system, like a templating system where we could easily make a new prompt to find variables or inputs to those prompts. And have it automatically create a a prompt to be executed within open AI or some other LLM.
So we call it like our prompt builder tool or like prompt builder system. And it combines exactly what I said, templates, data sources, variables that all come together into a prompt that can be executed. And we take that response, whether it be plain text or format. And convert that into different pieces of interact quiz.
So that took a while to do, like, you know, I think. Even in this couple of months that we’ve been working on it, we’ve had our own engineering improvements and also like the open AI and just LLM ecosystem has improved massively as well in terms of providing more consistent response output, i. e. you can say, hey.
Answer this question, but make sure you put it in a JSON object or like a JavaScript object such that we can parse it immediately and make sure to use these key names when you’re describing it that didn’t exist six months ago. And now it does. And we’re like immediately incorporating that new feature, a new offering.
Into our system, so it’s been like, you know, overwhelming things are changing every single day and in this ecosystem, but we’re trying to keep up with it. And you know, incorporate those in. So, I’d say was a big thing. And then the last part is actually. Cutting down on the execution time, or I guess.
You could call it like prompt refinement. How can we make sure that these prompts are running as efficiently as possible and as quickly as possible such that we don’t have to wait. 10 minutes for a quiz to generate instead, it’s like, 5 or 3 or 2 or 1. That’s the end goal. So we’ve been looking on refining those and kind of improving the.
Like, execution environment of these prompts, such that it’s really, really fast. And I’m just visualizing Evans, like, really dumbing it down for the audience right now, but I’m visualizing them in an engineering team meeting, just using all these, like, terms that we have no idea what they mean, but they understand it, but we’re just over here.
Like, yeah, thanks for dumbing it down. No idea what you’re talking about. I have a question, Evan. So part of this, first of all, this year long process has been insane from beginning to end. And I almost like when I stopped sort of started taking back some of my time, I almost had like similar to like writer’s block.
I was like, Oh my God, what was I doing before AI came along? Like I forgot my In some ways, my roles and responsibilities, because I was so consumed by it. bUt 1 of the things that I, I think I asked you this when we had, like, a demonstration or whatever we want to call it. How do you guys. Keep up with all of these changes and demands and updates and upgrades from open AI, you know, and then customer feedback.
Is it just, I guess I’m just curious of the process behind the scenes, how that works for you guys. Yeah, great question. Something that we’re like constantly trying to answer. I think it, the word that popped up in my head is prioritization. Everything that we do has to be looked through the lens of like time criticality.
Importance benefit that is going to bring a user benefit that it’s going to bring our internal team, um, weight against the amount of time that it’ll take to make it. So, it also reminds me of, like, the, there’s an iron triangle of a project management, right? You have time. Resources and scope. And for any projects, you can change, you know, how much time you have, how much resources you have, or you can change the scope of the project.
But those are the constraints on which you get anything done. So, that is the way that I look at the prioritization and kind of like, stuff we get coming in. Do we have enough time to work on it? Is this like a scoped. Project or ask um, and do we have enough resources? Do we have enough interesting to work on it?
And we try to optimize that equation if you will, to deliver the highest impact. For the lowest cost for lowest scope or lowest resources, but there’s sometimes where, hey, you really do have to take the time to invest a lot of resources. bEcause it’s going to be a really big improvement. I think like the end to end automation of like generating quizzes is a good example of that.
It’s a super large scope project. It’s going to take a lot of time. It’s going to take a lot of resources, but the end benefit is really, really high. So we invest in it. And that’s kind of like our framework. We, we have a variety of stack ranks uh, like a spreadsheet of all the features we could be working on that have details kind of about the 3, 3 different pieces.
I mentioned how long would this take? How much resources we have to put towards it? What’s the scope of it? And then what’s the actual end user benefit. so We have that in terms of features. We have that in terms of bugs of things that are reported internally and externally. We always think about it in terms of that lens, probably annoyingly.
So I know most people here, we ask, is there a workaround for this? Like how critical is it? And I know it’s can be an annoying question, but it’s one that we always keep in because. We’re super limited, you know, we’re a bootstrap team. We’re startup. So we just as much as we want to, we don’t have the ability to work on everything.
So we have to be pretty ruthlessly prioritized and focused on what we are are doing. Can you honestly, that brings up like, Oh, sorry, I was going to ask if you could just give a quick perspective on like, how big the engineering team is. I think we always talk about how our small team there’s like, is there 13 of us now?
1213. But of engineering, I’m sorry, of engineering, exactly. How many of you are working on the interact AI? Right now we have two. Excitingly, we hired our third full time engineer. Four, including Matt. And then we have Jared, junior dev and like QA tester. So we have a engineering team of five. And then two.
Myself and Philip working full time on like AI related features, small, small, but mighty team. Well, I was going to exactly say that I was going to say when we look at it in retrospect, we’ve been doing this for a year, but we are literally doing this such a small team. Evan was taking a lot of it for a long time because Philip just got hired on.
So he’s really been doing this a lot with maybe some help with, you know, Jared and Matt and stuff like that. And it’s like. Insane when you look back and look at the process, and it’s been taking a long time, but it’s also a lot of work, you know, from 1 person for 2 people. And I now understand since this is my 1st learning experience with this whole process, why a lot of big companies can push out things in 3, 4 months because they have insane teams and they can do all of this work in high volume.
And we just don’t have that luxury. So it’s a lot of respect. Hat goes off. Yeah. Yeah. I was gonna ask Evan, like as an engineer, is this like, because of this AI boom this year, is this the first time you’ve worked with AI? Like, was this exciting for you? Was this like new? Like, just because when they throw us in marketing into like a new software to figure out like, you know, it’s definitely a learning curve.
But I mean, Was this the first time that you’ve been working with it? Yeah, since the, the last time I worked with AI was in college taking a fundamentals of AI, like machine learning class. And that wasn’t actually using AI models that was learning. The math that goes behind making AI models, which was really hard and really confusing.
So this is the first time that I’ve actually, you know, I guess worked with like AI models doing prompt engineering out in the wild. And yeah, it’s, it’s super exciting. I hadn’t really played around with AI before working at interact and having used tools like. You know, chat, GPT and llama too, and all these different models.
There’s super, super powerful. Like you can automate away a lot of the boring stuff. You can augment your own workflows, not completely replace them, but augment what you do. In a way that allows you to focus on the more important stuff, like, creative aspects, like. Interpersonal, like, brainstorming and like, high level company planning.
While not worrying about, like, you know, the small stuff, which I think is super, super cool. I would say. It’s overwhelming to like, we’ve, we’ve talked about, like, just the rate of change and keeping up and how you have to prioritize what you adopt and what you don’t. buT in a way that’s made us, you know.
More more prepared for change. We know how to evaluate. A new feature coming in and decide whether or not to use it. Love that. I always think I was like, were they as overwhelmed as we were or like blown away by what it can do or was this like new for them when we first started? Oh my god. Yes. I felt like, I was like, oh, so this is like Siri or like Alexa, right?
I just say, hey, Alexa, do something. And now that we’ve learned more about it, I’m like, okay, no, this is way more than that. Yeah. I, I’m thinking back to Damaris and Evan, you guys weren’t here yet, but Jesse, you probably remember this when we were at the Russian river on an offsite and we were already kind of talking about this idea of taking someone’s, um, I don’t know, like whatever website, their business, or like putting it into, or like taking like a PDF and then putting it into something where it organizes the information, like it would have had to be coded, but like, Would it be possible to take their information?
And give them suggestions for quiz, but back then I wasn’t really at least like, what it is now wasn’t here yet. And so, even though this was all new and exciting, this was like, something we had thought about doing something like for a very long time. So it is just really cool to see how far it has. Come and without some of the manual work of like coaching, sadly, Jackie’s internet stopped working and is not here to talk about it, but with all the work of coaching, talking to customers, Demeris and customer support and social media, like that all put in sort of the foundation to get you guys started with the AI.
And then, you know, as we obviously were building it. You shadowed everybody in, in, like, what that process is like and using it. And then that kind of helped sort of bring it to where it is now. Yeah. That kind of reminds me of like, you all had this idea for almost like an AI related system. Years ago, but the, the tech wasn’t there, you know, that’s like reminiscent of Netflix when they wanted to start doing video streaming of like, we want to bring movies online, but internet bandwidth speeds just weren’t there yet.
The technology was not there, so this, it was a very valid idea, but just couldn’t be supported by the current infrastructure. Like we, we kind of had the same, you had the same type of experience. Where the idea is there, the preparations there is a team, but you need some type of like tech breakthrough to allow that to happen.
And I think that’s definitely been. You know, that’s what we’re seeing today in the, kind of the era of AI generation. Do you think, I know, do you think that like, let’s say hypothetically, I don’t know, open AI crashes, right? Like what happens to all of the platforms that are being built to like call upon, I don’t know, is that what you say?
Like it makes calls to open AI to like, Produce the output. So, what happens? Can I curse on this show? It would be bad is the short answer. And like, what you’re asking is a super relevant question. And one that we’ve another thing we’re starting to consider is like, how do we decouple ourselves? From any single one LLM provider, because right now we’ve used it a lot.
We’ve said it a lot on the show already. We use open AI. We mainly use their models, but ideally we’d have a system that isn’t tightly coupled to one provider such that if one goes down, we can immediately swap in another one, or if. The quality of 1 goes down, we can swap in another 1 or do some testing and see which 1 is best for our system.
And I think that really came to the forefront when, like, all the stuff about open AI came to light about, they were pushing out their CEO and then he was back and he wasn’t and then they were going to move to Microsoft with Sam Altman. Still going to be there or not. We, like, as a team, we’re like. Oh, this behemoth company that we thought was, you know, only going to grow, grow, grow might not be doing that.
We might not be able to depend on them. So it put into perspective that, you know, we have to adopt new technologies, but we do also have to have a contingency plan. For if those new technologies or dependencies go away, we have to prepare ourselves as best as possible. I was just going to say, do you think that, like, open AI is ever going to get to a point of, like, Google, like, we’d be in the same position as a company if, like, Google crash, you know, or Google drives or email the world every crash.
Yeah, like, do you think, is that where it’s leading to? Like, that these AI models are going to I mean, so many things are going to rely on them. There’s so many businesses that incorporate AI now. Yeah. I think they’re quickly becoming utility status, things that are going to be, we’re going to consider them so important and so ingrained into society that if they were to go out, that’s like your electricity going out or your water going out, the implications of it going out are.
Not necessarily something you can gracefully recover from, like, for example, we have systems where after somebody completes an action we send them an email, right? We have some automated hook. That’s a dependency. We use a system to automatically send those emails after they do something that you could consider a non critical dependency.
If it’s fail, that’s all right. It’s a best effort service, but. If a core part of your functionality or product starts going down, that’s that’s a critical failure. That’s something that you can’t gracefully recover from because it’s, you know, completely messing up a really, really critical workflow. And, yeah, with how how much is grown and how much it’s being integrated into.
Every single product or like software that’s out there on the market today, it’s becoming much more of a critical dependency. Can you quickly clarify for listeners? Like, what, what part of the product is pulling from open AI and depends on it? And then what part of our product is just interact? Yeah. Yeah.
Great question. So, cool. Open AI is only used on like ai. tryinteract. com for the actual. Initial creation of some quiz, uh, that can be put into your account, the copy, the copy. Yeah, you know um, but the builder, the dashboard, the analytics, everything beyond that initial creation is still. A human using software, or some user you know, making use of that software to edit their quiz or look at analytics that does not have any type of AI dependency or use AI in any way right now, just good old fashioned software.
Yeah, and I think that’s important because I, I’m sure people, I don’t think a lot of people realized that a lot of these platforms that, you know, you fill something out and then you get content using AI, it’s built on top of a, like, a tool, like open AI or any of the other ones, it’s not like they coded the AI themselves.
Although I guess maybe they did. But I think it’s important to differentiate, like, if. It were to go down, it wouldn’t be the whole product that goes down. Like, you could still go in and create a quiz. You could still go in and use the template. It’s the content creation part. That would sort of.
Struggle if something were to happen exactly. So I really want to say this this quote, you guys. Do you guys know Marvel movies? Are you guys a Marvel, Marvel people? Yes. A little bit. Okay. The Glorious Purpose. Where is that from? Ugh. Ooh. God. I’m not that good. There’s so many. There’s so many. There’s so many movies.
You guys. Season 2 of Loki just came out from Loki. I haven’t, I haven’t finished it. The Glorious Purpose. Hold on. I haven’t finished it. Okay. I’m still on like episode 2. No spoilers. Oh, dang it. I ruined it for you. Anyway. The Glorious Purpose is. To get better and, you know, integrate this. I just want it to be easy.
It fails.
Now we have to put a spoiler alert in the disclaimer of this episode. It works out. Yeah. If I could get on my soapbox a little bit, I think like that clarification, like brought up a point of like, what does have AI and then like, what’s still human driven and like. Those 2 ideas, especially today are like, in such tension, I’m like, where is that going around?
What, what do people want to have automated away? Where do people still want human touch or to know there’s a human behind it? And it’s a, a billion dollar question, I think, and it depends on the product, the service, the person. The goal that you’re trying to achieve is how much, what do you want to automate away?
And what do you want to know that there’s a human person still behind? Yeah, and I think what I’ll give AI or interact like a little credit here is that we’re trying to market this um, and incorporate these features into our product in a way that just augments. Existing workflows, but doesn’t completely replace, you know, a human creating a quiz or actually publishing it at the day.
And I love that because I think that’s so important. Yeah. Yeah. I feel like we’re giving AI like a good rep. Like it doesn’t, you know, in the past couple of months, it’s kind of been like, Oh, AI is bad. Like human connection. It’s like, why can’t there be both? There totally can be. Yeah. Yeah. You don’t want to be like.
And even like. I may just be no, let me not say that. Nevermind. Just kidding. There were 2 lawyers in New York that wrote a legal motion and filed it in court. That was completely generated by chat and they were either find or disbarred. I forget which 1 both as they should have been. But that’s an example of, like, a dumb use of chat, and then all the examples of students, you know, writing an essay and completely copy and pasting it from chat.
There’s really dumb ways to use a, they’re really dumb ways to use any tools out there, but. We, you know, as stewards or people that are providing this platform can make it a not dumb usage a way that augments existing workflows makes you more creative makes you better at, you know, creating a quiz.
But it’s not completely taking you out of the picture or using things without considering or looking at the output. For getting at the end of the day, I actually encourage people, like, to think of, like, our tool or a tool as it’s a rough draft of your quiz. Like, just think of it as a rough drive. It’s getting you started and you can then elaborate as a human would your thoughts.
The copy you want to add, make it your own more like, more specifically, you wouldn’t want a computer to do that for you. You would want to do that yourself and make it. Unique to you, so it’s really just a tool, like Evan says, to get you in and get you started. Take the thinking out. I think Josh has used the phrase like the blank page problem is becoming less and less of a thing.
You won’t have to stare at a blank page. You can be like, hey, help me generate some initial ideas. Help me get off the ground. So I can, you know, start running with some idea and even when I use AI sometimes, and I use it to help me generate ideas. I’m like, I don’t like any of those ideas, but reading these gave me better ideas.
So, thanks. It’s still, yeah, exactly right. So, if you ever create a quiz and you don’t like what our AI gives you, I guarantee it probably gave you some better ideas, some things to compare, contrast and go from there too. I love it. So to close this out, Evan, can you give us a sneak peek? Not visually, but I guess with words of what’s coming for Interactive.
Yeah, so the sneak peek is really the first version of that end to end automated quiz creation process, and that will look like a form that you fill out as a user where we ask some specific questions to try to get to know your business better. And the purpose of your quiz, like the goal of your quiz, um, in ways that don’t use, I guess, you know, quiz jargon, like, what’s your outcomes?
What are your results? Where are you trying to bucket users? Like, we don’t do that. We try to keep it in terms of your business, the problems that you’re helping people solve. And what your business does, and then we’ll take those and kind of convert it into that quiz structure. So some form will be filled out.
Then you can create or link your. Interact account, um, and after that quiz creation happens automatically in the background. Where we take the answers to your question, fetch more data about your company using user provided links or websites, and put that together into an interact quiz that is then automatically added to your account.
And you can open up in the dashboard or the builder and edit the colors, the fonts, the color you know, the cover photos, tweak the wording, like we were just talking about, you know, add a little bit more of your own voice. And then publish it and allow people to learn about your company and interact with you.
Unintended. Yeah. That’s our goal. That’s insane. Incredible. Join Royal. To be determined. TBD guys. But you guys can still check out our Interact AI, what it is today, if you head over to ai. tryinteract. com and get started there and play around with it. So Evan, thanks for dishing all the deets today and hopefully Jackie’s okay with her internet.
But guys, thank you for joining me again this week. And for those who are listening. Let us know if you have any questions on AI, any ideas, any feedback on a quiz if you’ve already requested one, and so on. And we’ll see you next time! Bye guys! Bye!