Eric (0:13) Hey there, and welcome to the office of the IT guy, the show where we celebrate the people and talk about the technology that's changing our world. (0:20) I am your host, Eric, the IT guy, Hendrix, and my mission here is to share a love for open source and help build a stronger community. (0:26) I'm so thankful that you tuned in because the office is now open. (0:49) Hey there, and welcome to episode three of the IT guy show. (0:52) I'm your host, Eric, the IT guy Hendrix, and I'm really excited to be bringing this topic to you today. Eric (0:57) It's huge. (0:58) Everyone's talking about AI, but I think there's so many things that could go wrong. (1:03) But in the right hands, I think there's so much that could go right. (1:07) Now in case you're expecting to hear Natalie from Shipyard today, had a last minute reschedule, so we're we're working to reschedule with Natalie here soon. (1:18) But we'll be talking about ephemeral environments in the not too distant future. Eric (1:22) So I was really excited that I had a couple of prerecorded episodes. (1:25) So really excited today because we've got a huge topic and a very passionate guest today. (1:31) His name is JJ Asgar. (1:33) He's a developer advocate at IBM. (1:35) And, not just any developer advocate, he is one of the members of the Instruqlab team at IBM. Eric (1:42) A little bit of background before we dive in. (1:44) IBM IBM developed a an AI called Instruqlab and partnered with Red Hat because they wanted it to be open source. (1:58) So IBM, of course, owns Red Hat, but but IBM came to Red Hat and said, hey. (2:03) Look. (2:04) We've got this thing. Eric (2:05) We wanna open source it. (2:06) We want it to become kind of a standard and and to be able to to be able to to open source it well. (2:15) And so Red Hat's entire business is built on open source. (2:19) So now Red Hat and Instruqlab at Summit this past May talked about talked about the open sourcing of Instruqlab and how it's designed to be not just an open source LLM, but all of the data, all the processes is going to be open source. (2:35) So you can actually go to instruct lab on GitHub, and we'll have the links in in the show notes. Eric (2:40) But the idea is that AI will be better, it will be safer, it will be more powerful, and more secure if it's open source. (2:50) If you've followed me for any length of time, you know that open source is at at the heart of my my technology dreams. (2:56) So I I definitely agreed with this. (2:58) So I I met JJ at DevOpsDays Kansas City, and he gave a very, very passionate delivery on Instruqlab, on on the future of AI in specifically, some of the dangers, but also some of the benefits. (3:12) And so JJ was kind enough to sit down with me at the event and kinda talk about, AI and what he sees. Eric (3:20) It was kinda funny that we met at DevOpsDays Kansas City because we kinda had similar senses of humor. (3:25) And the funny thing is he showed up with Watson x t shirt on. (3:28) I had a Red Hat shirt on, and we're like, we know each other ish. (3:34) So we we sat down. (3:35) We had a great conversation. Eric (3:36) So without further ado, here is my conversation about Instruqlab and AI with JJ Asgar from IBM. (3:44) Alright. (3:44) So I am joined with, JJ, and JJ works for IBM. (3:49) So kind of a kind of a distant relative of mine, as a as a red hatter. (3:53) Why don't you introduce yourself, and we'll we'll dive into today's topic? JJ (3:56) Sure. (3:57) Hi. (3:57) My name is JJ. (3:59) I'm a developer advocate for IBM. (4:02) I'm not a 100% sure what that means anymore, but I get to represent IBM in a lot of public spaces. JJ (4:09) I understand my privilege is rare, and I try to do the best I can for our community. Eric (4:15) Awesome. (4:16) And so you and I actually were in person at an an event in Kansas City DevOps days Mhmm. (4:22) 2024 and very quickly realized that we had a lot in common, a lot to talk about. (4:27) So, against his will, I actually drug him into the speaker room and stick stuck a microphone in his face, which he was more than generous to to pick up and and, agree to talk. (4:38) So one of the things that is running rampant in in our industry right now is this concept of AI. Eric (4:45) Yep. (4:46) And I I agree with your assessment that AI and artificial intelligence are two completely different things. (4:52) Yep. (4:52) So why don't we start out just by defining our terms? (4:55) What are when when we say AI, what does that bring to mind? JJ (4:59) Yes. (5:00) So what I've learned, very, very quickly speaking in the public about this stuff is that, first of all, everyone thinks that AI is what we had in movies, and it's not. (5:14) So what we're having to do is re level set what expectations of this new technology is being created and what it can actually do for us as humanity. (5:26) I like to start a lot of these conversations with also saying that you'll hear me only ever say artificial intelligence maybe once or twice to reference that other thing. (5:35) Mhmm. JJ (5:35) Because what we have is AI, which is not intelligent. (5:40) It is the best librarian or yes man you'll ever or crony, whatever you wanna say, that you'll ever have in your life. (5:48) It is just trying to figure out what you need and, to a 100% of what it's looking for to be able to tell that story and answer those questions you give it. Eric (5:58) So when you're when you're training an AI model, you're you ask it a question, it provides information, you give it feedback as to usually, it's a a thumbs up or a thumbs down. (6:07) Mhmm. (6:07) So when you say it's the best yes, man, it's it's that it's that training response of what was what I provided you accurate? (6:14) Was it what you were looking for? (6:16) You give it a thumbs up. Eric (6:17) That's reward. (6:18) And so the the model itself almost learns in that situation. JJ (6:23) So you just you just showed your your hand a little bit there. (6:26) Uh-oh. (6:27) That interface of the yes and no or thumbs up and thumbs down is done by one of my competitors. (6:33) That is not actually a standard way that we we tell AI what it's doing. (6:38) And that that competitor is the one that was first to market and kind of laid the land and made the expectations for everybody now. JJ (6:47) And it's amazing how quickly, when we go out into the public, everyone assumes it's a chatbot or it's a way to Right. (6:56) Communicate back and forth and and have that conversation. (7:00) But the AI space is so much more. (7:03) It has a whole world of good things that it could do for humanity. (7:07) It all can do some really bad things for humanity. JJ (7:09) And part of being technologists and actually engineers in space is that we need to learn every little choice we make will have ripple effects in this space going on gates. (7:21) So we have to slow down. (7:23) We have to take a breath, hit those brakes. (7:25) Because, frankly, if we make the wrong voice, conscious bias is real. (7:31) Mhmm. JJ (7:31) Bias is real. (7:33) And before we know it, we might actually you know? (7:36) I have kids. (7:37) Right? (7:38) I want my kids to be in a better society than I do. JJ (7:41) And the technology we build here, if we start implementing them that we want to, it it could get dark really fast. Eric (7:48) And I I think we'll touch on that a little bit, and I kinda I kinda derailed this, with saying too many words. (7:53) So we've we've got this of AI, which is not art intelligence. JJ (7:57) Yes. Eric (7:57) Now one of these terms that usually comes up is an M. (8:00) And how does that fact this picture? (8:03) So JJ (8:05) LLM, which audience, to make sure they understand, is a large model. (8:08) Mhmm. (8:09) It is, it's natural language or NLP or language processing to interface to figure out thing that you would like to get done. (8:18) So the chatchis of the world, I'll just call it because you're because everybody knows what it is. (8:23) When you talk to, like, hey. JJ (8:25) Give me a markdown give me a a table. (8:31) All the city or the the capitals in The United States Mhmm. (8:35) As table format for me. (8:36) It it it what you asked for and then outputs that. (8:40) Mhmm. JJ (8:40) A large language model. (8:41) Do that type of work for But as you start going deaf, do so much. (8:47) A thing called context that can very real very important very quickly. (8:51) It needs to understand what you ask for. (8:55) And as you start going down farther into the space, you quickly that there are strong limitation in the AI space start really engaging as a whole. JJ (9:04) One of the I I talk to people very quick is Like, why why else would I conference. (9:13) Right? (9:14) Right. (9:14) And I've been up into a few other very free and open source. (9:19) I was an IRC on free note in the nineties. JJ (9:22) But what I'm trying to make the the the point is one of the biggest challenges, just with, like, that we live in right now, versions of software, a little bit more polish around them. (9:32) They're because you're trying to to do that stuff and build off of Right. (9:35) Well, if, well, you already should know. (9:38) But audience, if you didn't know, they're all meant for open source AI, And it's real to recognize that AI is real. (9:44) We give you that data center to be able to actually say actually know how this from the very beginning. JJ (9:49) So unconscious biases, those Biden or, you know, praying that only this press president on the planet, right, I'm not I'm not gonna call me. (9:58) Right. (9:58) As you start looking picture of the world, need to have be able to see that that frankly, the correct way. (10:05) I know correct is important. (10:08) There are competitors that are not in the open source that will never give us those. JJ (10:11) And it is very important, especially for our for our economy. (10:16) Yeah. (10:16) And then those data set through open source allows you to get open source code. (10:21) But is the source code of a model? (10:22) I mean, it's not a horrible 100% true. JJ (10:24) Okay. (10:24) I mean but for the share of the conversation, the the the was trained on, the model that's or the data the models today, first code. (10:34) And to the data, it's done. Eric (10:37) So that's that's great direction of the conversation because that's how you and I started talking. (10:42) Year on I had a red hat, so it was, you know, conversation. (10:45) Never in my career seen be in a position like it is. (10:49) There is we've being back in my career, there's a little virtual. (10:53) There was green. Eric (10:54) There was there's all these crazy last I interviewed Laura, DevOps and ideas and says it's like a position that you hire for. (11:03) With this AI move is very much that we have to get right. (11:07) And as anyone that knows me will I have been Linux and open. (11:11) Mhmm. (11:11) That's been my career, years now. Eric (11:14) I mean, it it goes when we had and I looked at vacation exam book for soft windows book right then and there. (11:20) Nice. (11:21) But comes to mind is to lead that charge community. (11:25) I'm I'm a bit biased. JJ (11:27) Sort of Eric (11:27) biased at the giant red fedora. (11:29) The Red Hat is probably one of the leading com by extension, IBM. (11:33) AI space we're not alone. (11:35) You know, it's not REM trying to trying to build AI and slapping a label on it. (11:40) Our open source models suited to that are training. Eric (11:44) Any that you'd like to call out specifically? JJ (11:46) Well, I call out that specific note. (11:48) You're getting me to Eric (11:49) Far too many times. (11:51) You're you're picking up all my Exactly. JJ (11:52) So I mean, you've been part of these things. (11:54) But that one thing that we should audience just to make sure that they level set on this. (11:58) They they understand the of what we're about to try to talk about. (12:01) Mhmm. (12:01) There's a a website out there that you called huggingface.co. JJ (12:04) I want your audience to your thought in their mind. (12:09) Kingface is the GitHub AI. (12:11) When you think of GitHub, you think of, you know, the way people PRs around or whatever. (12:15) People have code out there that you can put on your laptop. (12:18) Hugging Face, a place for AI data and models and data and things like that. JJ (12:23) And to recognize, you go to huggingface.com and start poking around. (12:27) Oh, it's just overwhelming. (12:28) Right? (12:29) Because one of the biggest problem, open source AI right now, anyone can take an open source what about on fogging face? (12:35) Mhmm. JJ (12:35) We'll have to give it back to the actual quote unquote source. (12:40) And they can fork right off of it or train or do something else. (12:43) So if you will there. (12:46) Attic, because if you start putting this in your biz would there someone, frankly, are doing work for you when you have Right. (12:51) Seems like you. JJ (12:53) So that's which is the hope that I have the privilege of being with research. (12:59) And just recently, a whole hog of run. (13:02) Yep. (13:03) And last week, of course Eric (13:04) At at the time of recording, it was Summit. (13:06) Uh-huh. JJ (13:08) We are building a tech of knowledge that we have a exactly what we put on from a continually make in, OAP flow through PR. (13:20) Yes. (13:20) I know. (13:20) I'm sorry, Morgan. (13:22) Through through the workflow, for a good. JJ (13:24) You're probably wondering, public good? (13:28) Suspicious already. (13:29) Yes. (13:29) Well, the operating system out there called Fiddler. (13:32) Right? Eric (13:33) I'm I'm vaguely Okay. JJ (13:34) Good. (13:34) Good. (13:34) Good. (13:35) No. (13:35) It's been around a little while. JJ (13:36) I think it's I think it Yep. (13:37) Yep. (13:38) Just released There you go. (13:39) See see up here. (13:41) Yeah. JJ (13:41) So what is Fodor? (13:42) Right? (13:43) A premium operating system that I don't think called RHEL. (13:45) Yep. (13:46) Right? JJ (13:46) It makes a lot of money off of. (13:48) Right? Eric (13:48) I've seemed to feed my family through Red Hat. JJ (13:50) There you go. (13:51) Good well, it turns out extremely well. (13:54) Right? (13:54) Where you work in the in the open, you come down to product. (13:57) Can I assist now? JJ (13:58) I mean, what is Fedora? (14:00) A public good of an operating on pretty can think of. (14:03) I don't think Spark supported it anymore. Eric (14:04) I don't think so. JJ (14:05) Well, I mean, who who's if you're if you're nerd running Spark. (14:08) That's the majority. (14:11) Truck lab is with the Merlinite granite model. (14:14) Mhmm. (14:14) We are we're training those in the found in the upstream. JJ (14:17) So, of course, eventually, other biz at and to run actual but you actually things in to the model. (14:25) New new knowledge is right now, Wikipedia, but the idea eventually take document or a policy from from government each to model these things, be able to have it all these side of it so governments can answer questions. (14:41) I pie in the sky. (14:42) I haven't quite actually gotten to work yet, but, The US has all through congress. Eric (14:48) Yes. JJ (14:48) They're all available for everyone to get those. (14:50) I'm thinking about once a week or something like and then using my model questions about all those Mhmm. (14:55) So we could actually have a public this model of these questions these questions about the bills that are coming out. (15:00) What is the actual budget thing? (15:01) Actually, read 600 pages of Eric (15:03) right? (15:03) I can't get some of our sys 100 dog because of RHEL nine releases. JJ (15:07) You see exactly understand where I'm going with this. (15:09) So so we're trying to do this in the public. (15:11) So we take away the what's a good word for it? (15:14) The the the the whole but I do want training these models. (15:18) Right? JJ (15:19) Who can run this stuff on their own, fortunately. (15:22) To train the model right now requires a Mac, which unfortunately have a lot of people out there, expensive, on some so it's but gaming PC in a very long I would like to I wanna achieve to get budget build gaming PC that would be of the InstructLab. (15:37) And I was like, I might as well it was money. (15:41) It's depressing. (15:42) But the point of this is is we're public. JJ (15:44) So we actually have resources that we'll get the community of from IBM, and I hope that makes sense. (15:48) I went kinda Eric (15:49) No. JJ (15:49) You're Eric (15:49) fine. (15:50) Anyone that's listened to me present ever know kinda do the same. (15:52) They knew what they were getting into. (15:53) And I like this approach, and and I'm a redheader. (15:57) But because I believe this is in between, RHEL, say, and a downstream product duct product. Eric (16:05) So got got words in order. (16:08) But so all this is available, and I want you to highlight the fact that there's this. JJ (16:12) Mhmm. Eric (16:13) The first, what are you and then the second is the data. (16:16) Is that data JJ (16:17) So we have two now that we are working recording today. Eric (16:20) It'll probably be different tomorrow. JJ (16:21) Oh, well, I came sitting down with you. (16:24) It was verified that the end end run is running as we speak. Eric (16:28) Nice. JJ (16:28) Which is an op IBM. Eric (16:30) Which will be I'm folks are are hearing this. JJ (16:33) But if you go to hugging fee I b m Granite Slash 0 m e, you'll actually set a model right there. (16:39) The Merlinite model, hugging slash instruct slash Merlin, a small Merlinite. (16:44) Just just Alright. Eric (16:45) We'll check the show notes. JJ (16:46) There you go. (16:46) Perfect. (16:47) And you will actually see the rebuilt in a third cadence. (16:52) We have some of the initial goals that were in some some conversations, but we we have rolled those cadence back because it should be hard, man. (17:02) This is this was a we so we're going to hit. JJ (17:05) But Mhmm. (17:05) Right now, we we everything will happen. (17:08) It will happen. (17:09) We're percent going down all those paths. (17:11) Unfortunately, some engineering, we've hit some friction in some other spaces. JJ (17:17) But that's projects. (17:18) Right? (17:19) Know that you this is not like, you don't really expect it to work. (17:23) And then, you know, first, I mean, I still remember compile X kernel back in the day. (17:27) Right? Eric (17:27) Yeah. JJ (17:27) When make menu can fame out, my life all of a sudden got help. (17:30) Right? (17:31) Oh, god. (17:32) I'm sold. (17:33) But in all seriousness, this is an open source project. JJ (17:36) Please, if your audience is interested, join Like, if you wanna be part of the and actually do some good We're gonna be slow. (17:44) Right? (17:44) And I if you just talk to my leadership, they're tomorrow. (17:48) And I'm taking a voyage of talking to and and really things down. (17:53) If we don't do this right I I know we gotta get understand that. JJ (17:56) I do. (17:56) But if we don't don't see in the world my kids are gonna grow up in. (18:01) And I Right. (18:01) Maybe this is the defeat. (18:03) Maybe this is I've just watched Fies of My Future, right, or whatever. JJ (18:08) I can see these Right. (18:09) With I see how this I can see the grind to do. (18:12) And I'm I'm not saying we can't do it. (18:15) We've instruct lab as the project out there at the time that is taking this the betterment of and just like in the DevOps, when automation starts taking people's jobs and people were when Ansible became so and I either way, but I I I understand pretty damn good. (18:33) Start taking people's job because they're like, well, I spent three days building a server. JJ (18:37) Well, if I you wrote this and and then built it for time in two and a half hour, that gave you two and a half days something more Right. (18:45) AI in that. (18:47) Imagine what we society when we're we're spending time things to everybody instead of three days against the wall. Eric (18:53) I love it. (18:54) And as as that sysadmin, three days domes and then you had to wait till we did whatever security did, there was something better I wanted to do, whether that was with techno automating or deal 200 tasks that I had. JJ (19:05) Did you ever have that, team to come in and all of sudden be like, hey. (19:08) So you got this got this beacon two fig thing they gotta run on this thing. (19:11) Do you have all the stuff I'm looking for? (19:12) Just say beacon fig two? Eric (19:14) That specific example, but definitely had the same problem. (19:17) Universities, man. (19:18) Universities. (19:20) It's like told me that. (19:21) No. Eric (19:21) Okay. JJ (19:22) Actually, no. (19:22) Just just a quick joke. (19:24) This is true. (19:24) This is I once when I was working once had a customer. (19:29) We wrote a ChefPuq to install a Puppet the Puppet local local with a Puppet models, the b config to we could check-in to another thing. JJ (19:40) So you're using Shuppet to talk to b config to somewhere else? (19:44) And they're like, yeah. (19:44) That's exactly what we do. (19:45) Why? (19:47) Because the you know, like, but, like, write a chef cookbook to do this whole thing about, like, standardizing on Ansible. JJ (19:53) So I'm like, why am I in this? (19:54) Why does it make any sense? (19:57) It was it was a couple weeks. Eric (19:58) I I don't doubt that. (20:00) I'm also thinking we'll have to as as part of, like, a episode of all the horror story. (20:04) I JJ (20:05) you know, the funny thing I make them up on the spot. (20:07) No? (20:07) I I Inventive. (20:08) Right. (20:09) Because I can't be that inventive. Eric (20:10) I can't princes I've gone to. (20:11) How many where the routine is you sue in the morning, bridges, and Mhmm. (20:15) Far too many notch doing the same thing. JJ (20:17) Yep. Eric (20:17) Fixed horse. JJ (20:18) Yep. Eric (20:19) And back when I was a sis admin That's JJ (20:21) true. Eric (20:22) Many good and so involved in if you're not involved in this group or anything, some of my best call our our conference family see a few times a year, but, it's a beaten track. (20:33) But JJ (20:34) Oh. (20:34) How you you find kindred speak? (20:36) One of the reasons for advocates are as a as a community ask and genuine I can talk about actual the career path and then but I do wanna mention a who's actually felt the talking about. Eric (20:49) Yes. JJ (20:49) Any developer advocate never never actually done the work. (20:52) They're just marketing marketing mouthpiece. (20:54) They have the title of developer advocate. Eric (20:56) Getting mouthpiece. (20:56) I find that in JJ (20:57) Oh, yeah. (20:57) You know? (20:58) Hey. (20:58) You got the term developer. Eric (20:59) Actually, no. (21:00) I'm I'm just JJ (21:01) Oh, nice. (21:01) Okay. (21:02) Oh. Eric (21:03) Instead of instead of OpenShift. JJ (21:05) So you should I I Eric (21:05) pick. (21:06) Actually, presents from Chris Short. JJ (21:08) Yep. (21:08) Oh, that's awesome. (21:09) That's awesome. Eric (21:09) Two episodes together and then said, oh, by the way, this is JJ (21:12) Somebody says that's that's that's a podcast. Eric (21:16) Nice. (21:16) Perfect. (21:16) Perfect, man. (21:17) An outro. (21:18) But I I'm a developer, and I write crap. Eric (21:21) If if someone ever comes up with an operator, it will be the Hey. JJ (21:25) If you can You're a developer. Eric (21:26) The program in YAML now? JJ (21:28) Well, you know, I I that project has taught me so much that I can I can faster than my winter? (21:33) This like, I I go into this, like, a few seasons. (21:36) I'm like, oh, this this this and that. (21:37) I gotta put going here. (21:38) Because, like anyway, this we'll talk about that later. Eric (21:41) What JJ is trying to do on people, it's not that hard to Oh my god. (21:44) So many so many. (21:47) Well, JJ, is there, like, to send for you to follow you on on social media? JJ (21:51) Pretty just looking for JJ. (21:52) J asghar, a s g h a r. (21:54) Pretty easy to I do have the email address @ibm.com. (21:57) I really do. (21:59) So if if you survive in I'm an open book. JJ (22:02) Just drop me an email. (22:03) I mean, you can find later and all that stuff. (22:04) Most of the most of the frankly, you're I know you are. (22:08) And I am too. (22:09) Don't worry. Eric (22:10) It's safe Safe space. (22:11) Virtual space. JJ (22:12) Safe space. (22:12) Please reach out. (22:13) There's if I can never help you, so please find it. Eric (22:17) Awesome. (22:17) I love that. (22:18) Well, JJ, it's time. (22:19) We're to have you back on the show real soon. JJ (22:20) Thank you, sir. Eric (22:21) Welcome back. (22:22) I hope you found that exciting. (22:24) I know just listening to it during, during, post editing, I just I got really excited about AI and the future of the industry again. (22:33) JJ is a very powerful speaker. (22:37) It it I don't think it did him justice in the audio, but you could you could see on his face as I was interviewing him just the passion, the concern with with the future of of the industry. Eric (22:50) It was really incredible to get to talk to JJ. (22:53) I hope to have him on the show again soon. (22:55) Hopefully, have a technical demo lined up because my hope is that the IT guy show is not just talking heads. (23:02) It's not just me or not just me and a guest. (23:04) But my hope is to also include, like, technical demos and that kind of thing. Eric (23:10) So part of this is just building back the audience. (23:12) Another part of it is just kinda getting all the streams going, but also had some really great conversations over the last couple of months, a lot of which came from DevOpsDate's Kansas City. (23:23) So maybe I should go to another another conference here real soon. (23:26) So definitely subscribe to our YouTube channel. (23:29) And if you're listening to the audio podcast, make sure to hit subscribe. Eric (23:32) That way you get notified anytime I release new content. (23:35) And until then, feel free to join my Discord. (23:38) Check out the show notes for additional links. (23:40) But until then, thank you so much for joining me, and we'll see you again real soon.