Pete Goddard:
When you have a board meeting and you're trying to figure out how to run your company, you want to put that front and center. We want to make our customers happy. It may not maximize short-term dollars, I have no idea. That's a discussion for someone else. But since we have moved towards community development, there has been significant benefit across the board.

Eric Anderson:
This is Contributor, a podcast telling the stories behind the best open source projects and the communities that make them. I am Eric Anderson.

Eric Anderson:
I am joined today with Pete Goddard of the Deephaven Open Source Project, and I suppose we would call it the Deephaven Company as well, is that right, Pete?

Pete Goddard:
Yeah that's right Eric. Thanks so much for having me. The company's called Deephaven Data Labs actually, but the project is Deephaven.

Eric Anderson:
Fantastic. So Pete, level set with us. What is Deephaven?

Pete Goddard:
Sure. Deephaven is a query engine that's built from the ground up to be excellent with realtime data first, so it really had that use case in mind from the get-go and all of the engineering is built to address that realtime data or dynamic data of any sorts, both alone and quite importantly in conjunction with static data since all that historical stuff provides a ton of context and is super important to realtime as well. So it's that query engine plus it's a series of integrations and experiences that we've developed to make users and use cases really fly with the engine.

Eric Anderson:
So Pete, maybe to unpack some of what you've described there, I imagine a lot of people are familiar with a data warehouse and well actually you focused on streaming data where I think a lot of people assume data warehouses are kind of static, right? You put data in and then you can request it out. But Deephaven is very focused on data in motion or realtime, is that the right place to start?

Pete Goddard:
Yeah. Deephaven was born at a capital markets company. We can probably talk through that history in just a little bit. But Wall Street companies tend to care a lot about realtime data, so streaming data and streaming infrastructure was important to us. From the get-go, Deephaven embraced a concept of streaming tables because we wanted the realtime capabilities of streams to be married to the intuition and usability and set of libraries that are generally available to tables. So we built up an application and evolved it into a product and then spun it out to open source, we can talk through that journey, but at its core, it's an API to support streaming tables alone and in conjunction with batch and then an engine that can do stuff on top of it. Architecturally it's hitting the basics that I think your listeners might be quite interested in. It's a no SQL data system, a query engine that's column oriented in nature, and is really organized to address data both at high density as it's coming in as well as certainly at scale for static sets.

Eric Anderson:
Got it. No, that's fantastic. Yeah, and then let's do talk about ... Because I think you have an interesting background and you have approached this problem by working with ... I don't know, the most demanding of use cases for this type of thing. I'm curious how Deephaven came to be.

Pete Goddard:
We have a unique story. It's been a real luxury to listen to some of your historical podcasts and learn the journey that some of the other founders have been on, but it also sets up Deephaven up to have a pretty contrasting story both for the company and how it's come about as well as for me personally as somebody that's able to interact with you. My history is really in capital markets trading. I was an engineer at school, but I went into quantitative trading back in the mid 90s when that was a little more avant garde than it is today. I was always involved in derivatives, always in modeling and always working with teams that were trying to do interesting risk management and do market making and short-term predictions and things like that.

Pete Goddard:
I was fortunate enough in 2004 to be part of a group that wanted to put a new company together, so a handful of partners started a company called Walleye Capital. Surprisingly probably to your listeners based in the suburbs of Minneapolis, Minnesota, and the short story is that company is really where all of this started, and I think there's a few steps along the way that might be interesting for you and I to explore because it does end up at open source and community development but through a path that's really quite different than others have taken.

Eric Anderson:
Maybe just as you're at this juncture, so at the outset of Walleye Capital, was the plan to be kind of very software-centric? I guess that's how you've been in the past? What was the kind of role that you imagined software to play at Walleye?

Pete Goddard:
So Walleye Capital was founded originally more or less to start from scratch an options market making business, another relative value trading business which means certainly not an investment type of business but rather a modeling business to try to buy cheap stuff and sell expensive stuff and have really good infrastructure for access to the marketplace. So you can change the profile of friction costs in trading in the market and maybe even move them all the way to a positive friction rather than a negative friction.

Pete Goddard:
My partners and I had been involved in that business at other organizations, so we were quite familiar with it. That business invariably couples two capabilities that will be very familiar to you, one is excellence with computer science and systems related to computer science, both on the software as well as the engineering front, and then the second is marrying math to that. Certainly a lot of quantitative modeling and a lot of machine learning techniques played a role, but also just solid mathematical and software infrastructure for managing risk in realtime.

Pete Goddard:
And one thing that I think is interesting and probably influenced me and the people around me was we had a ... As I said, a focus on options and other derivatives, and the universe of options is about three orders of magnitude larger than the universe of stocks and futures. So essentially you're held to the latency profiles of stocks and futures but you have a much larger universe to manage and pricing derivatives is as you'd think, like it's a tail on the dog, so there's a lot of math that has to come to the fore at low latency and scale so it makes for a really, really interesting problem set for a computer science and quantitative team. That's where we started the company.

Eric Anderson:
Certainly, and you described this as kind of a unique story versus some of the other shows we've done. But at the same time, I'm seeing the parallels that like Kafka was born out of LinkedIn, they had a big log processing problem and Hadoop out of Yahoo and anywhere that people have a big problem I think is an opportunity to build a pretty awesome system, a solution. And I feel like you were sitting on one of those at Walleye.

Pete Goddard:
I certainly share your sentiment and point of view on that, but it's not lost on me that it's awfully generous to put Walleye in the same category as some of the behemoths you mentioned or have led to other data products. I think it is an interesting part of the story and I do think Wall Street infrastructure, Wall Street software, Wall Street math, has a whole perspective and universe of capabilities that is a little bit outside of the typical hyper scaler and Silicon Valley approach to data and infrastructure and application development. So it's really been exciting over the last many years to somewhat segue from one to the other.

Eric Anderson:
Got it. I assume Deephaven started there and you built it for Walleye and then at some point it kind of grew outside of Walleye?

Pete Goddard:
That's right. So it actually didn't start in '04 because when you're really dealing with high frequency, a general purpose data platform, which Deephaven is, isn't particularly relevant. But in 2011 or 2012, we're looking at our business and relative value options trading and market making generally ... It's an arms race business. So the competition had dwindled to a fairly small number of players, Walleye amongst them, but we were truly shoulder to shoulder in the electronic pits against the bulge bracket firms that would immediately come to mind if you thought of who were the big trading houses in the U.S. and to a certain extent a couple of them in Europe.

Pete Goddard:
So as the CEO of the company, I had conversations with my partners about how we wanted to invest going forward and what we wanted to do with our team and whether did it really make sense for us to be working on FPGAs and trying to make sure we had the best microwave services to take advantage of time and place between different locations, or was that just simply a gain that had to be left to larger players and we should do something else with our talent.

Pete Goddard:
And though we remained committed to that enterprise and evolving our capabilities, we made a strategic decision to engage in sort of phase two of the company, where we looked around that table and we thought, "Hmm. We have some pretty good mathematicians, we have good software, we have good infrastructure people, what else can we do with that." And we decided to step out in the direction of a platform hedge fund or quantitative hedge fund generally that could have lots of different teams pursuing a variety of pursuits across the world in different securities and that the company in the middle would provide capital risk management and systems and it was at that time in 2012 where we took a look at the systems we had and we realized, "Well we needed a general purpose data system," and it was at that moment for reasons that might be worth learning more about, my partners and I decided to invest in the development of the application that became Deephaven.

Eric Anderson:
Awesome. That does seem like in some ways an engineer's dream to have kind of this carte blanche opportunity to build an awesome system from the ground up for like I said a high powered use case. So you went to work and at some point you realized this general purpose system would have greater utility outside of Walleye?

Pete Goddard:
We did what any reasonable company would do which is we did everything, we pursued every possible avenue to not build something. We actually thought a general purpose data system must be available, we just had never needed one in the six or seven years before that. We had some pretty basic requirements, we had 130 people at the firm, we wanted all of them to use it. There are lots of different types of people but they could handle anything from Excel to the most sophisticated quant thing to some of them were really hardcore developers, others were more portfolio managers or some SQL people. So we wanted everyone to use it, we wanted to be great with realtime data and historical data. We didn't think realtime meant hard, hard, high frequency realtime, we thought realtime meant milliseconds, that would be fine, and we wanted it to be great at standard manipulations of tables and data access but we knew we also wanted to deliver code to it directly. So at the time that meant mostly Java and C++ to us and we looked around and frankly there weren't any systems that checked many of those boxes. So we didn't really have a choice. If that's what we wanted, we had to roll our own.

Pete Goddard:
So as you can imagine and certainly this is the case of lots of the other founder stories I've heard, you can't take for granted that just because you want to build something that you can. I think I was very fortunate to be teamed up with some truly capable computer scientists, they had some breakthrough moments, some real aha moments driving in the car or hanging out in the bathtub, I don't know exactly where it came to them, and they were able to ... It turns out solve some problems that were pretty unsolved in the world, [inaudible 00:12:16] unifying streams and batches in regards to making some of the architecture that was going to be needed for this application actually go, so it was a very exciting time.

Eric Anderson:
Pete, I have an appreciation as you know for the debate on kind of streams and batches and the combination. That's something the team of mine at Google worked quite a bit on and I agree. I think you brought some novel solutions to the table here which has been exciting.

Pete Goddard:
Yeah. It's certainly exciting to be a part of it. I have to say with appropriate modesty, first of all, all of this technology is the work of a team that had a mandate and was resourced, but they deserve the credit. I certainly was not the one solving those hard problems. The other thing that's interesting about this is when you survey the data systems that exist today, like a lot of them grew up parallel to one another, right? So frankly the fact like, "Oh, there's your team," for example, I know of that team, but I didn't know of that team until about two years ago and two years ago I'm reading some of their material that they're putting out in the open in regards to these open problems or the way they look at things and it was really exciting because you got to see, "Oh, hold on, that's the way we look at it," and we've kind of solved that problem.

Pete Goddard:
But everyone's got work to do and we're developing on our own, not relying on Google to for example provide the solution for us. So we were in some ways naïve to what others were building while we were building things and I think we just feel very fortunate that many of our decisions seem consistent with what the community wants and we're very fortunate that the customers we've had have led the product in a way that we think is extremely modern and relevant.

Eric Anderson:
So bring us up to the present day. You solved several of these problems and what next?

Pete Goddard:
Yeah, so this is ... Like everything so far is kind of generic, right? I've been telling you the story of here's a trading firm, they built some tech, cool, lots of trading firms do that, not a big deal. This is where it gets kind of spicy and interesting. After the product was used for about five years internally, and you can imagine it evolved quite a bit, starting from a small team to then being maybe seven engineers working on it. The engineers and I had a meeting that was sort of timed with our lives in such a way that we determined a few things, one, that the product did really revolutionize the firm, that the company was able to diversify and grow in ways that we thought was pretty effective and we felt that there were certainly lots of reasons for that and lots of people deserve credit, but we felt that there was a role in that success story for this data system. So we determined that, one.

Pete Goddard:
Two, we made the conclusion that the product was relevant beyond Walleye and we hypothesized it was actually relevant well beyond capital markets. We saw in late 2016, early 2017, that data that changes matters, that batch was not going to be the only part of the future, that a no SQL database that could bring code to data was really empowering for a lot of use cases and those use cases were not going to be fringe, and that we had something that would be really interesting outside of Walleye and outside of the capital markets. And this happened to be timed with a point in my personal life where I'd been in trading for a long time, trading has a unique set of things you have to do and the way you ... Stresses so to speak. I don't find it particularly stressful, I think that's a normal way to think about it, and my job had evolved a little bit away from systems into other aspects of running the trading company, and I was interested in getting back to systems and software and my love for the Deephaven platform. So the engineers and I decided to spin it out into its own company, which is one half of the story for how that came to be.

Eric Anderson:
Wow. Yeah. So now you get to noodle on engineering to your heart's delight, right?

Pete Goddard:
That's right. Yeah, my job changed and I got to work with computer scientists and people that care about systems exclusively from that point on.

Eric Anderson:
Now I guess the project shifts too, where before you had a very clear mandate, you had one customer, a well-defined use case, and now you have to go find customers and kind of broaden the use case description I suppose.

Pete Goddard:
Yeah, I think that's right and that's potentially putting it mildly or maybe even being generous. I think the phrase ignorance is bliss might come to play here, where certainly we weren't the first capital markets company that spun technology out and thought they would bring it to the world, but there weren't a ton of success stories there, and I think one of the reasons for that is people like me think that they have a good product and they can bring it to market and it will just be a fit and the customer service is easy. Remember, traders don't have customers. It's like the only business in the world. You go to a career fair and they always ask you about customers and trading businesses obviously at a bank or something like that they do but not in the open markets, trading businesses don't have customers. So it's weird.

Pete Goddard:
So there was a lot to learn as we brought the product out, and as you know, because of your history, there's a big difference between I have a product inside that I'm developing that I get to tell people that are on the payroll, "Hey, use it, and that feature you want, it's just not important," and they just have to swallow that, versus doing that in the outside world, where you're either prospecting, selling or serving a customer.

Pete Goddard:
So we're fortunate that we had good counsel about being realistic, about finding product market fit and investing heavily towards that and being focused from the get-go on just making sure our customers were successful with the platform, even if that meant rolling up our sleeves in any way needed.

Eric Anderson:
And at some point, and maybe I'm jumping ahead here a little bit, but at some point you had to decide about one element of the go to market being open source and was that a consideration as you spun out a company or when does that come into the picture?

Pete Goddard:
No, it wasn't a consideration at all. We spun out and all the IP and the team was spun out. Walleye became a customer, we went about the business of acquiring a number of other customers that were not lookalikes of Walleye but other serious players in the capital markets doing a variety of things. Open source was not even a conversation at that time. It wasn't until about two years ago that my developers and I at Deephaven started to lean pretty heavily into the notion that we'd be much more effective if we were developing in the open, if we had community products and if we could really move in that direction. The problem is we had an enterprise product that was underpinning some very serious operations. We had an investor group that needed to be turned around on the idea of open software which they really just heard as free software, and we had a code base that we really had to think about in regards to APIs, packaging, and support for the community. So we had a lot of thinking to do to make community development a reality.

Eric Anderson:
It sounds like your developers came to that conclusion. They were like, "If we open up, we maybe have a better appreciation for how we fit into the broader tool set, and maybe it improves the product."

Pete Goddard:
I'm a fan of your podcast and as I listen to it I think what I surmise is you're really trying to assess how does this come about for everyone. It's so interesting, right? And there's all these different stories. From our point of view, moving towards open source, though it took some work to get all the constituents on board, was a really good answer and yeah, it's our developers who came to that conclusion first. They knew that it would be the answer for interoperability which is really, really vital. It's not even vital, it's almost interoperability first. They also knew it would be probably our only effective strategy for engaging verticals beyond the capital markets. We did what any reasonable software company would do. As a VC, you might advise them, "Hey, don't go fishing all over the ocean. Just start off the pier that you know, right?"

Pete Goddard:
So we were selling to and supporting companies where we could not only talk tech but we could talk use cases, but we knew that, "Oh, just because we think Deephaven is exceptionally relevant for IOT and healthcare telemetry and realtime gaming and energy and power and crypto and blockchain and all these things, even though we think that, we don't know jack about those industries." So the way our developers were very keen to let the community become aware of the product and its differentiators, its very unique capabilities and try and bring those and its integrations to industries well beyond the ones that we understood.

Eric Anderson:
I think the interoperability first is a big point.

Pete Goddard:
There is one other side to this though and that is ... And I think I'd be remiss if I glossed it over and that is the role of an investor who invests in not open source software that needs to be convinced that open source is an interesting path and I think in our case there were two things that were vital there. The first is there's some real success stories and thank goodness for them because they've blazed the trail that the rest of us can follow and at least the rest of us can point to that there are models that work, right? The Mongos and the Databricks and the Elastics out there. So it was really important to have those success stories that went ahead of you because you could point to them and say, "Look, I don't even know that I know all the arguments but it can work out."

Pete Goddard:
So that's one, and then two, I think the other thing that was important in our case is remember we had been in an arms race business before, we think we've done well but we understood the costs of an arms race business and what felt like the fragility of an arms race business. Like you always have to be working on your weapons. It's not a very nice analogy, but that's the way it works, and we didn't like that aspect of commercial software, that there's all this innovation out there and you have to be scared of it rather than embrace it.

Pete Goddard:
And then the last thing is really just from a selling perspective and I think this is vital to open source is even just in the four and a half, five years we've been doing this, the buyer has changed where now it is their expectation that all the good stuff that comes with open source or community development, that they can benefit from that. From, "Hey, they want a POC," without getting their legal people involved, to they want to know that integrations are coming, they want to be able to see the community involvement and all the support tickets that the community has gone through, and they love the transparency of the code being there and at least the option to extend it themselves. Whether they actually do or don't is immaterial. So it really just hit on all cylinders, it just took some time for us to find religion.

Eric Anderson:
I still face some of that. Less so now, but when I started in venture four years ago, I wanted to focus on open source and I felt like I still had an uphill battle with other VCs who thought that was ... I don't know, not crazy, but they were like, "Well, that worked for Mongo but now there's AWS and everyone's kind of afraid of AWS." And everyone was doing the license changes to kind of carve out space from Amazon, and I think you're right that a lot of people don't quite even know how it gets from A to B but they're convinced B exists and so they're willing to kind of take the road of A.

Pete Goddard:
Yeah, my dad was a salesperson for most of his career. He was not a typical salesperson and he to his bones cared about his customers. Like he just wanted them to be happy. And I think when you have a board meeting and you're trying to figure out how to run your company, you want to put that front and center. We want to make our customers happy, and there is significant benefits. Significant benefits to having open development processes, having the invitation for interoperability, and there has to be the right payoff that will just come by putting your customers first. It may not maximize short-term dollars, I have no idea. That's a discussion for someone else. But in the longterm, you want to innovate, you want to stay on that wave. It is hard and you want your customers to be happy and that's where we are right now and I can tell you unequivocally, though we have real challenges in making it happen because of our enterprise commitments, even with those challenges, since we have moved towards community development, there has been significant benefit across the board.

Eric Anderson:
Pete, I'd actually like to also talk to you a bit about the product and its role in the kind of data infrastructure ecosystem. I feel like we have been talking about doing streaming and stream processing for a long time in the enterprise and it always feels like it's just around the corner but I kind of feel like it's actually just around the corner this time.

Pete Goddard:
Yeah. I think it's here. I maintain that there will be an explosion of use cases and a change of habits because the tooling around it is easy to use and so powerful that it just changes your mindset. Frankly, again I'm not ... In contrast to almost everyone you've had on the show, I'm not a computer scientist, I'm not a transactional SQL person, I'm a CEO of a trading company historically, right? So I look at data systems as ... Look, I want everybody to use it and I want it to do analytics and AI for sure, but I also want it to support applications like, "Guys, just make that happen, I want it all." And with that mindset, capabilities with streaming technology is really, really vital, and from our perspective streams ... One of the things we always get caught up in is ... I always get two questions, always, always, always, every time I'm prospecting or talking to somebody is they will always ask me, "Yeah, but you're like a high frequency guy. Nobody really cares about that type of latency," and I immediately say I totally agree.

Pete Goddard:
Data streaming is not about latency, some people need low latency, others don't. Streaming technology is really just another phrase for data that changes and from my perspective that's really where the story is, that your architecture needs to embrace the fact that hey, your data's changing, so that should probably be a fundamental first principle, rather than a bolt-on or something you're always trying to work around.

Pete Goddard:
That's the first question I always get. The second question is always aren't you like blah blah blah, and there's 20, 30, 40, 50 technologies that will be thrown into that sentence, and the answer is invariably yes, that there is tremendous overlap between capabilities in different systems, Deephaven overlaps with many of them, but we have significant differentiators that give us reasons to exist.

Eric Anderson:
What do customers of Deephaven get excited about that can help inspire some of us to give it a look?

Pete Goddard:
So from an architectural point of view, from the get-go we've had what's called an incremental update model. You had a recent guest on that has another system that has a similar technology, theirs is much more around transactional SQL. The incremental update model just means like hey, you have a table, and then you have new data coming in, whether it's appending or changing the structure of the table. Batch says well you got to look at that table and now do your calculations again. An update model says hey you could just keep track of the changes to the table and keep track of your calculations so you're doing an incremental calculation rather than a full recalc and for many workloads, that proves to be both much better performance and much lower in terms of resource utilization and it really makes it so that both for simple use cases, even ones that non-developers can approach, as well as for complex use cases that have many steps, that it just empowers capabilities that just can't be done any other way.

Pete Goddard:
So at the core of Deephaven, that update model is vital and is a huge part of the story. There's lots and lots of other things going on, lots of other problems that we've solved, but that's probably the place to start, combined with we do that in a way where you unify stream and batch, whether you're working on one of these streaming tables, one of these tables that change, you're working on a static table, you as a user, you as a Python developer, you as somebody writing table ops, you as somebody who's using a C++ client that is employing Deephaven as a microservice, you don't have to care about what that table is doing in the background, whether it's static or updating. So that's a real luxury I think.

Eric Anderson:
To ground what you're saying a bit if I understand it, if you have events or you have data that's being streamed, you can transport that data from A to B and that's one technology I think Costco does a lot of if you want to call it transport. But if you want to reason on that stream, you generally have to aggregate it. You have to look at the data relative to the rest of the data which is to say if I want to know the average or even the highest in a window, that's a statement of how that data relates to the data around it and that requires ... Like I said, aggregating the data, and what you're telling us is that you can do that aggregation once, come up with an answer, and then the next time you want to update the data, you don't have to redo the whole aggregation, you just do a much smaller calculation to tell us the delta that's changed between the aggregation then and then the one with the new data.

Pete Goddard:
Yeah, you got it exactly right. Let me just throw a couple of details in there. One, Kafka is very exciting. Our users employ Kafka heavily. They also use other Kafka compatible providers. There are a few that are also quite interesting and have good technology, so I wouldn't want to starve them of attention. We think Kafka streams and other event streams are really interesting. We actually support or have made available in community, but with an Apache license, an API for streaming tables. Goodness we think it's great, many usage patterns should just use Kafka or other distributions of event streams or things of that nature, but we actually support an API that allows you to publish tables and table changes.

Pete Goddard:
We've delivered it to the community as a project called Barrage. It's not a heavily used term, but that is a word that falls on the back of Arrow, right? So I think of it as an extension of Apache Arrow Flight, so Arrow is a project started by the Dremio people and Wes McKinney, at least as I understand it, for in memory table, sharing across applications, sort of without copying in performant ways and they extended it across the wire with Arrow Flight and Barrage essentially is this open source package that allows you to add metadata to their payload that describes table changes. So now all of a sudden, you have tables being shared across applications but also moving across the wire but these tables don't have to be static, and we think that that's quite an important innovation that we're really excited to bring a community around because we expect there is a significant community that cares about data that is changing, that wants to use tables just natively, even though they're not batch, and as partners, they'd like to extend the interoperability with that API.

Eric Anderson:
I already feel like that's easier to reason with than like you said, raw events, which I think are the typical kind of streaming medium.

Pete Goddard:
Sometimes I ask people like, "Hey, do you ever think in JSON? Like can you draw it on the board?" Like it becomes difficult to think about events and JSONs once you get into sophisticated use cases like many of these formats. But it probably is just that Excel and maybe SQL but probably Excel taught everyone how to think in terms of tables and they're extremely intuitive. So we're excited about the potential of that API and it's all built on top of GRPC, right? So it really is quite enabling from our perspective and when you think of use cases, one of the things to consider is oftentimes when people talk about query engines, they tend to focus on table operations and we care about those a lot, right? All of the classic relational use cases, "Oh, I want to aggregate, I want to join, I want to filter, I want to sort, I want to do partitions, et cetera." But we very much think about time series as a first class capability and we think that's really important to many, many industries that data doesn't really exist, like data timestamped at the time that the data was relevant really, really matters in a lot of your structure.

Pete Goddard:
So Deephaven is organized to be excellent not just with relational type of stuff but also time series manipulations and time offsets and time map and all of that. But then even more importantly, a data system has to go well beyond queries. Like I'm always trying to figure out what other systems can do and I hear so much of the vernacular is around table operations and that's very important but Python Pandas made clear there's a whole user base that just wants to bring code to data and Deephaven from the ground-up was built with that use case in mind, that we want to be able to deliver code to data, have it compiled down with the table operations and have things work. We want TensorFlow to work as a first-class citizen, you don't want a client server model where you need to do table operations and then deliver it to Python. Like we want it all to be available at the CPU and run on the data directly.

Pete Goddard:
So Deephaven is organized to be able to do all those table operations and all that time series stuff on batch and streams but also to support these sophisticated use cases where you're bringing other languages to bear and frankly most of our investment in the last few years in that regard is to make Deephaven a Python first experience. It's a Java application at its core, so it's good with Java, we have lots of C++ use cases, we're very open to lots of other languages in the future, but we want it to be a very, very Pythonic experience from a deployment perspective and from a usability perspective.

Pete Goddard:
One of the real benefits of being on open source is I think we take very seriously our engagement in other projects and we think very much about the way we're delivering software so that it is valuable to as many people as possible. So to the second point, we have broken up the Deephaven project into sub-modules that we think are natural breaks. So we have a repo around something that we call our WebUI and I would encourage people that may not be interested in our data engine but are interested in delivering either big data to a browser or via their own means outside of Deephaven, realtime data to a web browser. We have a standalone package that has lots of dashboarding infrastructure that solves some very interesting problems, and in particular has a table grid that we're quite proud of and allows for the display of almost infinite data in a browser which is a hard problem, it's an unsolved problem we think before we put something out. We just wrote a blog piece this week that says, "We think everyone else struggles with two or three million rows. Once you get to there, you struggle, and we document that we can support a quadrillion rows in a browser."

Pete Goddard:
So there's all sorts of challenges around that and we focused on making sure that that component is available outside of the data engine, other competitive data engines, other companies could use it, and we very much welcome that. And then to the first point I was saying, we're trying to be good partners with other projects that we're benefiting from and trying to extend them, right? So I talked about Arrow Flight and we have a nice conversation going with them, they gave us the constraints about how to load these table dynamics into their payload, we thought that was really interesting. We're working with the GRPC around a proxy servlet called Jetty that is helping us solve problems related to delivering streaming data to browsers which is really quite difficult. Once you get into streaming see there's all this infrastructure that it solved for batch but it's really not solved for streaming.

Pete Goddard:
So we've been working on that and then the last thing is there's a company in Europe who spun up a really interesting bidirectional bridge between Java and Python and we were early beneficiaries of their work. This is the cool part about open source, right? Like they're working on environmental informatics and stuff like that, really far away from the type of use cases and industries and verticals we've been involved with, but they had this bidirectional bridge, it did most of what we needed it to do because as I said, marrying Python and Java is really important to us. So we've become co-managers of that project with them, and have extended it ... Certainly we think to the project's benefit although obviously to the benefit of our use thereof. So these highlight some of the things that we think are so compelling about open source and have really invigorated the way we're approaching software and its development out in the open. We're trying to follow the example of some of the real heavyweights out there that are as engaged and as open as possible in the way that we're managing our project and developing software as a team.

Eric Anderson:
I think it's neat that some of these other kind of open from the start projects never quite get to the full stack thing. It's a core piece of technology and then there's examples of other elements on top, but because you've delivered an enterprise application, and then unbundled it into the open, I think you get these other elements. Like the UI you talked about and the grid work, I saw that blog post. So I think it's kind of a neat benefit of your unique model.

Pete Goddard:
Thanks, yeah. I think our team of developers I'm fortunate enough to work with are thoughtful people and take very seriously essentially the culture that we're trying to establish with the community. So we look forward to the development that will come and the dialogue and direction that we hope the community provides.

Eric Anderson:
Thank you so much Pete. We've covered a lot of ground today between how this all came out of your capital markets work, the journey through open source, and the capabilities of a product, or really the platform and several open source projects that you've got incubating. It's very exciting stuff. Can you point us to how folks can get involved if they're interested?

Pete Goddard:
So the project is out on GitHub. We have a couple of organizations there. One is GitHub Deephaven, the other one is GitHub Deephaven Examples. We have many examples that we release as standalone projects where you can download a docker image and everything should just work. We're a younger project relative to some of the other data infrastructure companies that you talk to, so we have not yet spent the time to itemize issues that we think would be relevant for contributors, so I think I would ... It behooves people or I would direct people rather instead to reach out to us if they're interested in contributing, just reach out to us on Slack or establish a GitHub discussion and we'd be happy to engage that way. But obviously we see tremendous opportunity to get involved just from a use perspective and from an integration perspective and Slack and GitHub remain interesting ways to get off the ground there. Our intention is for things to be very easy regardless of the deployment model that you want to use, whether you're using Python or not, whether you want TensorFlow to automatically be packaged in the deployment, we want to make that easy. So we'd appreciate any feedback people have in regards to the startup process.

Eric Anderson:
Great. Thanks for coming on the show today and I encourage everyone to take a look because I think this is a bit underappreciated gem in open source. There's a lot here that is kind of largely undiscovered. Thanks Pete.

Pete Goddard:
We're just getting rolling, so I appreciate those kind words and this was a real treat. I'm flattered and I learned a lot. Thanks so much.

Eric Anderson:
You can find today's show notes and past episodes at contributor.fyi. Until next time, I'm Eric Anderson, and this has been Contributor.