Featuring Legends of Learning
Read Time: 15 minutes
Welcome to Episode 58 of Students vs. Startups. This week, moderator John Gilroy talks with Sandy Roskes who is helping to teach kids through interactive video games built by Legends of Learning. These games help kids learn in a fun way which leads to improved participation and overall retention. Listen below to hear about their unique learning concept.
Thanks to our Sponsor:
John Gilroy: Welcome to Students vs. Startups, showdown at the Potomac. My name is John Gilroy, I'll be your moderator today. Big round of applause for show number 58. Yeah! Al Gore. Thank you, Al Gore. Send you a birthday card or something for inventing the internet and allowing us to do these podcasts. If you've listened to these before you know we've kind of taken over a room over at Eastern Foundry, the conference room here, kind of a big old table. One side of the table, we've got students. One side of the table, we've got a startup. We have a little 26-minute conversation. We walk out of here fast friends.
John Gilroy: One side of the table we have our students. Let me introduce our students, we have Ekaterina and Elias. Ekaterina, your background please.
Ekaterina: My name is Ekaterina. I'm a student at Georgetown. I'm halfway through my technology management program. I also work as an IT consultant. In my role, I oversee federal IT portfolio. I'm involved with application maintenance, development, enhancement, modernization for their applications and websites and associated cloud platforms.
John Gilroy: Interesting. And a lot of cyber security applications, too, aren't you? Elias, your background please?
Elias: So, I'm also a Georgetown student at the same program, tech management, and I did my undergrad in electrical engineering. I'm currently a software developer at Accenture, also working on a side gig, a startup for real estate.
John Gilroy: So, his goal is to be on the other side of the table. He wants to be the startup, maybe happen in a few weeks. And our startup today is kind of an interesting company. It's called Legends of Learning and the representative is co-founder and CEO, a gentleman by the name of Sandy Roskes. Sandy, how are you?
John Gilroy: You know, my daughter is a big proponent of this anime and cartoons. I go to your website, legendsoflearning.com. All kinds of ... a fun place to start and a fun place to learn. Tell us about your background and this company, please.
Sandy Roskes: Yeah, sure. So, thank you for having me today. I am a serial entrepreneur, so I've started a number of different companies. Also, have a background in venture capital and as you noted early in my career, my training was in engineering, so I worked as an engineer for a few years before moving towards technology and startups.
John Gilroy: And so what business problem does Legends of Learning solve?
Sandy Roskes: Yeah, so at Legends of Learning, we believe that teachers are superheroes but even heroes need good tools. So, Batman has his tool belt, heroes need great tools. What we found are the core insight we had in starting the company was that teachers, educators are still struggling, their core problem is how to engage kids in learning, how to get them engaged, keep them engaged. It was the same when I was kid, it was the same when my parents were kids, and it's true today. So meanwhile, classrooms look very much today like they did when I was a student several decades ago. Some changes but the basic format of a classroom is the same, but meanwhile our lives have been completely transformed outside the classroom. We all ... we shop differently, we buy differently, we order food differently, we research things differently, we interact with the world completely differently. So, our idea was to bring the same kinds of advances that have taken place outside of the classroom into the classroom and in so doing, engage students in learning.
John Gilroy: Well, one of my taglines is "I put the fun in fundamentals." But going to your website, you do put the fun in learning, don't you? It does kind of look kind of fun. The guys with capes, the different kinds of kids doing different things, it looks like a whole lot of fun, doesn't it?
Sandy Roskes: That's the idea, is to make learning not only appear to be fun but to actually be fun.
John Gilroy: Yeah.
Sandy Roskes: So we were, myself and my co-founders, we were all gamers as kids. We played lots of games, both online and offline. I grew up with Atari and arcades and we decided that gaming was a uniquely engaging kind of medium and so our idea was to bring games into the classroom but in a really structured and curriculum-aligned way.
John Gilroy: Elias?
Elias: Yeah. I'll start off with funding. How have you been able to leverage your venture capital background to funding for-
Sandy Roskes: Yeah, sure. So, we've been founded by ... we're funded to date through our founder and Angel money so we do ... we all have prior companies and raised quite a bit of venture capital in past companies and we'll be going that route as well with this one, but to date we've really funded it all with ourselves and with Angels.
Ekaterina: My first question would be about your idea, how you came up with this idea to integrate games into education.
Sandy Roskes: Yeah, well, so the kind of core personal insight was we learned as kids a lot from games. Games like Oregon Trail, maybe you've played, or Sim City, adventure games, which promote critical thinking skills. We didn't do it formally, it wasn't in our classrooms but a lot of what I understand from American history was from things like Oregon Trail. So, we kind of recalled that, we sort of married up the fact that there's a lot of technology enablement in classrooms today, there's a lot of devices, there's a lot of bandwidth. So, there's an ability to bring games into the classroom, so our idea was to actually do that, to sort of square that circle and bring the medium into the classroom now that you can do so.
John Gilroy: Elias?
Elias: How big do you think this market really is, in terms of gamifying education?
Sandy Roskes: Enormous. Learning is huge, right? First of all, it's not just K-12, you guys are in school, and in graduate school. We learn lots of different things. You can learn almost anything through a game. In fact, you think of the most highly trained people on the planet are fighter pilots. How do they learn to fly? Through games. So, you can learn almost anything with a game so really anything that can be learned, we learn all throughout our lives, the market is literally limitless.
John Gilroy: So, let's say I'm teaching science in Loudoun County Public Schools. So what I would do is, I would use you as supplemental or where would it fit in for a curriculum in a high school?
Sandy Roskes: Sure, so we do consider ourselves supplemental, as opposed to core curriculum. We actually do have some people using us as their core central curriculum, not because we intended to, but because we are comprehensive. So, right now we cover middle and elementary school science and we pretty comprehensively cover it and so it's ... we intend it to be a supplemental but in which anywhere in the scope and sequence you find content you could use it pretty much all year round and find the content you're looking for.
John Gilroy: Elias? Cool.
Elias: What are some of your key performance indicators that you use to measure success and do you use any BI tools or any type of software to just kind of draw that-
Sandy Roskes: Yeah, sure. So we definitely do, we're all pretty data crazed. Actually, before we started the company we did a large research project, which I can go into a little bit further, but what we use today to measure our performance is first of all engagement. How many teachers are on the platform, how many students are on the platform, how many games are run every day, for example. Things of that nature. This will be our first full school year actually in the market. So, we will start to look at performance on assessments, state assessments, but we haven't really had that data yet live in this school year but that'll be the next thing that will be done.
John Gilroy: Ekaterina?
Ekaterina: Who are your main competitors in the education space?
Sandy Roskes: Yeah, so if you look from the narrow question of what other types of games are used in classrooms today, you have large games, things like Minecraft, gaming platforms that have educational value and are used in the classroom. Probably the more direct competitor if you will, to us is really Google. You can Google ... if you're teaching Newton's Laws, you can google "games on Newton's Laws" and you'll get a lot of returns from Google and teachers do that today. They will just get on Google and look for gaming content. You don't necessarily know what you're gonna find, you're not gonna necessarily be sure it's actually a game or aligned with your curriculum but that is the kind of thing that people do, but what we're trying to bring is a comprehensive platform that has all the gaming content that you might be looking for. There's really nothing like that today.
John Gilroy: So Sandy, your target is high school, large counties or is it parents that want that supplemental? Who's your target?
Sandy Roskes: Yeah, so our initial target is really teachers in schools, they're our core user today. We try to acquire, if you will, teachers to use the platform and then we will sell to their school or to their district but there is also an avenue to really market to parents to really buy the platform for the students outside of a school context.
Elias: Could you elaborate on your onboarding process, in terms of is it per school and the challenges you may have had and some of the difficulties between acquiring a lead or a school or a "customer" quote-unquote as compared to retaining that school as a lead? Because as you know, apps turn a lot.
Sandy Roskes: Yeah. No, they do. So, you have to be really delighted with an application you've used in order to keep using it, so we designed it in a way so that teachers can find us through various means and really start using the platform on their own without any intervention from us. We do, however, when schools or districts acquire us more formally, we do training, professional development to help teachers get on the platform, to really kind of load them up and get ready to go and really train them on how to. But ultimately you have to ... teachers have to be able to find what they're looking for, be able to successfully use it in class and have everybody stay engaged with the application. Then they keep using it. That's what we find, if they get to that point, they become pretty repeat users.
John Gilroy: At 5:30 this morning, I drove by something called the Homeschool Legal Defense Fund and when I read about you, I said, "well, home schools would love this!" Also, parents may look at someone's curriculum and say, "well, my son Claude is in the school and boy, great science program. Maybe we can, in certain areas, maybe" ... and so, what they can do is take advantage of you for a school that doesn't have a certain strength and so I think it's really that you've got to educate that parent more than anything, I'm thinking.
Sandy Roskes: That is, that's coming around, as well. We really started with classrooms and teachers to show that it can be used in class and that it really was rigorously aligned. The next frontier for us really is parents and really to bring it into the home. We do have homeschoolers and home users today, plenty of them, but that's actually something over the course of the next year we're gonna start to really address, kind of home use and parents as our customer, if you will.
John Gilroy: Mr. Elias?
Elias: So, I think everyone really has their own definition of product market fit. How did you know what the signs were for you and when did you start to see that?
Sandy Roskes: Yeah, so I think you couldn't really have done this two years ago or three years ago, only in the last year or two could you bring this. The things that you needed to have in place were kind of ubiquitous devices, particularly in the classroom. Things like Chromebooks, Chromebooks were a big ... iPads, certainly, and Chromebooks, were big enabler and only in the last couple of years do you have sort of ubiquity there. You also need bandwidth, frankly, sufficient bandwidth in schools. Again, only in the last year or two.
Sandy Roskes: There's another, I don't know how far deep in the bits and bytes you want to get, you're an electrical engineer, but there's a game development platform called Unity, which has sort of democratized the development of games, made it much easier to bring a wide variety of games to market. That's also relatively new. So, these are sort of the pieces that were in place. The core need ... the need to engage and the difficulty to engage kids, that's sort of timeless, but the technology pieces have really only come into play in the last year or two.
Ekaterina: There are several ways to bring up technology into education, one of them is simulation. Is it something that Legends of Learning is interested in pursuing, is it in the scope of the company, and I'm talking about simulation ... let's say you are training the firefighter, right? So for them you need to have the sort of environment for them to go through and be the first responder.
Sandy Roskes: Yeah. So, in our case, we do employ simulations in the platform, we have simulation content on there. I mean, what we're not really doing is a deep immersive experience, if you will. We're not trying to have a fighter pilot cockpit and immerse students in this virtual reality world, for example. There are folks that are doing that sort of thing. We're trying to use classic game mechanics, the kinds of game mechanics that people are already familiar with and really build content ... games that employ science content within those game mechanics. That's mostly what we're doing.
John Gilroy: There's companies out there like Huddle that target the commercial world. Is that part of your future plans, too?
Sandy Roskes: Yeah, we will ultimately, our ... we're starting in K-12, we've more narrowly started with actually middle school science, that was our starting point and we're kind of edging out from there, building upgrades. We'll continue to build out subjects, full grades, and then go beyond K-12. Absolutely, and suddenly commercial corporate training. Like I said, you can teach anything with games.
Elias: So, I think education, in our country especially, is impacted by a variety of things. Especially, the economy and where you live in the country and some kids may not have the best resources and tools to get educated. How do you think you guys are trying to solve the problem or help cover some kids that may not have a mobile phone or an app or the resources to enjoy some of this-
Sandy Roskes: No, that's a great question. There's obviously a lot of inequities out there in the market so ... or just in the world. So, that's part of the reason we started with in class use. So, again, as I mentioned, the device ubiquity is really ... ubiquity is maybe a strong term, I wouldn't say there's 100% coverage of students with devices in classrooms but it's become very pervasive and that's including in schools ... you know, there's programs like Title I, for example, which bring dollars and hardware and internet to less advantaged schools and in fact, in many cases, those schools have better coverage with devices whereas they may not necessarily ... not everybody has the sufficient devices at home. Not everybody has a phone, not everybody has a laptop or a desktop at home.
Sandy Roskes: So, that is one of the reasons that we decided to start in the classroom, because that's where you kind of have the most pervasive presence of technology.
John Gilroy: Sandy, I noticed you're based in Washington, D.C., is that right?
Sandy Roskes: Yes, I am.
John Gilroy: Doing anything, maybe pilots, with the D.C. school system?
Sandy Roskes: Yes, we definitely have some deployments within D.C. and also in the surrounding areas, in Montgomery County, Maryland, Prince William County in Virginia is a very big ... we have a very big presence there. And then further out in Virginia.
Ekaterina: So, sounds like you're already piloting your product. What is the biggest challenge you experience so far with entering the education market?
Sandy Roskes: I think inertia, right? I think the biggest competitor is the status quo. Teachers are ... everybody's busy, okay? We're all busy. I don't know if anybody is as busy as teachers. School teachers are just phenomenally busy. Maybe John, you know that. Every day you come in, you've got another five to seven classes, you've got to have all those lesson plans, you've got kids, you're in front of the room, you've got papers to grade, you've got next week's plans. Teachers are just outrageously busy. So, I think the biggest challenge that we have is to break through that, really get noticed by teachers, start word of mouth, and then create enough excitement and joy and happiness with the platform that teachers continue to use it and tell their friends, start the word of mouth thing going.
John Gilroy: I've been reading about Ancient Rome here lately. My vocation is reading about strange topics and they used to use wax tablets back in Ancient Rome. And, you hear about Abraham Lincoln, he used coal or something, you know? And then I grew up with paper, and I think my grandchildren are going to move to something different. I think this isn't really that wild, it's actually using current technology in order to engage the current audience that it is.
Sandy Roskes: Absolutely, yeah. It's really leveraging ... it's meeting kids where they are, it's leaning into the distractions that kids have today, everybody's got their device, they're on Instagram or whatever. Using ... playing Fruit Ninja or Angry Birds or what have you. So, our idea was lean into that, that is a ... kind of this never-ending ... stop fighting these trends and bring content that actually ... swim with the tide.
Elias: So, what does the game ... the app do to incentivize continuous use for the student?
Sandy Roskes: Yeah, that's a great question. So, that's ... we're actually building in more and more functionality to generate that kind of feedback loop, reward loop that you get from games and gamification. Things like leaderboards and leveling up and all of those types of things. To date, what we've really had is teacher-directed play where teachers will find the games that they want their students to play in classroom and very much the next wave ... coming out over the next few months with things that really promote that reward loop and this ability for students to go back in on their on, and the desire for students to go back in on their own. Student-directed learning is another big trend. Teachers want students to be able to find where they ... go at their own pace, find the material that they want and building in that reward loop to encourage students to come back in is something that we're working on right now, actually.
John Gilroy: You know, I'm in the classroom every week, and in four weeks I'll be giving a lecture called "Measure and Monitor." M and M. And it applies to you. So, can you go in to an education institution and say, "if you use ours, then you will have a test grade increase of XYZ?" Or, what are the metrics you use to prove that it works.
Sandy Roskes: Yeah, sure, that's great. I kind of alluded a few minutes ago to a research effort that we did before we founded the company. We partnered with Vanderbilt University, which is almost as good as Georgetown. A very fine university, obviously with a very strong department of education and learning, and a cluster there that's particularly interested in game-based learning. And so, we partnered with them to really study this question. "Can you take a curriculum, chop it up into little pieces, build games around that curriculum, get it successfully implemented into the classroom, and then show an impact on engagement and performance?" That was the question, that was the study. Vanderbilt really ran the study, we brought the games, and what we showed was ... the kind of headline was a substantial improvement in performance on a post-assessment. It was a true academic study, randomized control trial, every teacher had a control group and game groups, and we showed over half a letter grade improvement on an assessment after three weeks.
John Gilroy: Wow.
Sandy Roskes: In one year of study.
John Gilroy: Wow.
Sandy Roskes: Particularly, there was an also an improvement on critical thinking skills, which was a really interesting result. It wasn't just facts. The assessment had multiple choice questions and also short-form and long-form answers, so we found an improvement in higher level thinking. We saw a particularly ... even greater improvement in performance amongst student with special needs. Full standard deviation improvement in performance versus their peers. And then, huge engagement increase, off the charts engagement increase. That was not necessarily surprising, kids playing games in class are going to be more engaged than probably anything else they could be doing, but it was a really dramatic improvement in engagement, as well.
John Gilroy: I have to admit, I went to Twitter, and I typed in @legendlearning, and it looks like fun. It looks like you could have sixth grade ... if you had a ... like, me in sixth grade, kind of a problem kid, you'd go there and you go, "I kinda... this looks like a cool thing, it looks like something I could engage in." Did you go to their Twitter feed at all, Ekaterina? It's really kind of a fun Twitter feed.
Ekaterina: I did, and I think it's very important to stay current with what's happening in the world, and that's the way kids learn, and you have to be able to accommodate that.
John Gilroy: Even for problem learners like me. Elias.
Elias: For your A/B testing, you had a controlled group and just a regular gamers group, and so how did your feedback from kids that liked games compare to kids that don't like games? How did that feedback differ and what type of conclusions did you guys draw from that?
Sandy Roskes: Yeah, that's a good question. So, just to be clear, there were about 1,000 students in this study, so it was a pretty large study. Every teacher had to have at least one control group and one non ... one game group. Kids and people loves games to a very substantial extent. It's not easy to find people that don't like games. Even people that are not real gamers ... you're not a gamer, but I gave you a game, chances are pretty good that you're going to get engaged in that game. So, there isn't ... we didn't really study this question, "okay, how much did you like games in general? How much do you like games in general and did that have an impact?" That was not a question that we really asked with this study. But, if you look at data, an enormous percentage of kids in K-12 play games to a certain extent. That's one of the reasons it's a very powerful medium.
John Gilroy: So, what kind of objections do you get when you present to a school system? What do they usually say?
Sandy Roskes: Yeah, I mean, there's ... you have some just skeptics in general that are ... "what, are you bringing games into my classroom?" There's a certain amount of skepticism.
John Gilroy: Old guys like me going "let's get out these books, sit there and shut up and read." That's boring.
Sandy Roskes: There's not too much of that, but yes, we encounter some of that. I think the main objection is more along the lines of there's just ... we have too many options. We could do this, we could do something else, if we're going to do this, it's got to replace something. The class period is still going to be only 47 minutes long no matter what. So, I think that's ... it's that inertia, status quo is kind of the biggest objection, if you will. But, more and more what we find is you might have a school where there's 10 teachers and some teachers adopt and some don't, because they're skeptical or they're laggard.
John Gilroy: And if they get results, guess what, they're going to brag.
Sandy Roskes: Well, that's the thing, they look over and they see these students playing games and they're engaged, you count 27 out of 28 students are on task, they're like, "maybe I should try that."
John Gilroy: Yeah, it's a winning record, it works. Elias, last question, please.
Elias: I'll make it a two-part question. First part is, if there's anything you could do differently, what would it be, and number two, is there a learning curve for teachers getting on the app as compared to students, and what are you guys doing to alleviate that?
Sandy Roskes: Teachers find it harder to get onto the app than students partly because students don't need directions on things, right? I mean, your average 10-year-old will just take any technology out of the box and start using it, they don't need a manual or anything like that. Not the case with adults necessarily. And also students are within the environment of playing games, there's less for them to learn and know, in a way, than teachers.
Sandy Roskes: Something we'd do differently, it's hard ... it's kind of early to ask that question. I think most of the key decisions that we've made were good decisions. We started with science, for example. No regrets there, it was a great place to start. Middle school, I think that was the right place to start. We've now moved downgrade. Maybe I guess from a content perspective, it's ... actually, we currently think that maybe we actually have too much content in middle school, we actually went maybe a little bit overboard in the number of games. We have almost 1,000 games covering middle school. We probably would be just as fine with half that number, and if we had more grades with the same number of games, maybe that would be a little bit better. But no main ... no big regrets, that this point.
John Gilroy: Well, Sandy, we are running out of time here. If someone wants more information, where should they go?
Sandy Roskes: Well, the best place would be our website, which is www.legendsoflearning.com. As you've mentioned, our Twitter feed is @legendslearning, another good place to learn. And, yeah, you can inquire right through our website and find a platform and get started.
John Gilroy: It does put the fun in fundamentals, it's a fun site. There's a lot going on there. If you would like to have show notes, links or transcript, please visit theoakmontgroupllc.com. I'd like to thank our founding sponsor, Reading Solutions. If you are interested in getting involved in geo-spacial projects, contact Reading Solutions. We are hosted by Eastern Foundry, a community of government contractors who are bringing innovative solutions to the government marketplace. More information go to eastern-foundry.com. If you'd like to participate as a student or startup, contact me firstname.lastname@example.org and thanks for listening to Students vs. Startups, showdown on the Potomac.