Podcast with Denis Rothman Part 1
Summary:
Unlock the captivating world of artificial intelligence inPart 1 of our enlightening podcast with Denis Rothman. Join us as we dive into
Denis's remarkable journey as an AI practitioner, from developing innovative
language teaching software to collaborating with renowned companies like LVMH
and Aeros Spatial. Discover the power of combining rule-based and probabilistic
approaches in AI implementations, and gain insights into the rise of large
language models from major players like OpenAI, Microsoft, and Google. Don't
miss this opportunity to explore the complexities and possibilities of AI as
Denis shares his expertise and experiences. listen and embark on an
extraordinary AI adventure in Part 1.
In the first part of the podcast, Denis Rothman, anexperienced AI practitioner, discusses his journey in the field. He started by developing language teaching software and later worked with companies like LVMH and Aeros Spatial on advanced AI projects. Denis highlights the coexistence of rule-based and probabilistic approaches in AI implementations. He explains the
surge in large language models and the dominance of major players like OpenAI,
Microsoft, and Google, but notes the cost and accessibility limitations for
individuals and smaller organizations. Denis emphasizes the need to combine
domain expertise and reasoning with large language models for more intelligent
outcomes. Overall, the conversation sheds light on the complexities and
challenges involved in implementing AI solutions.
[00:00:00] Andrew Liew Weida: Okay. Before, let me first introduce you like okay.
[00:00:03] Andrew Liew Weida: Thank you for coming to theshow. To anybody who's listening to this allow me to have the honor to
introduce our guest for today. His name is Dennis Rotman, who graduated from so
university and a Paris did university writing one of the very first work to
metrics and embedding solution.
[00:00:20] Andrew Liew Weida: He began his careerauthoring one of the first AI connective natural language processing or nlp. In
shot chat bots applied as a language teacher for Moit, Shaan and other
companies. He authored an AI resource optimizer for IBM and apparel producers.
He then authored an advanced planning and scheduling.
[00:00:45] Andrew Liew Weida: Solution use worldwide.Dennis is the authors of artificial intelligence books such as Transformers for
natural language processing. Of course for more info you can check out his
LinkedIn website and Dennis Rotman at zero B 0 3 4 [00:01:00]0 4 3. And of course I'll put it on the LinkedIn show notes.
[00:01:03] Andrew Liew Weida: And his recent publication,there's a lot, but I think of one of particular notes is the Transformer for
Natural Language Processing, building Innovative Deep Neuro Networks,
architectures for NLP with Python Pi, Python, TensorFlow, but Roberta and more
paperback on the 29th of January, 2021. So Dennis is, thank you so much for
allowing me to introduce you.
[00:01:27] Denis Rothman: Yeah. And thank you forinviting
[00:01:29] Andrew Liew Weida: me. Yeah. I would love toreally ask you the first question. I think let's let the audience know because
I think a few of our listeners. Actually contacted me and say, Hey Dennis
Rotman actually, let me put it up over here so that we can recollect the Okay.
[00:01:44] Andrew Liew Weida: That conversation. Okay.One second. So it's easier and more fun that we can talk about it. Yeah. So
that was the, let me put out the link LinkedIn post. So yeah, this was the one.
Yeah. Let me do a bit of
[00:01:57] Denis Rothman: zooming in. So which one is [00:02:00] it?
[00:02:00] Andrew Liew Weida: I think this was the onelet me read out. Can you see this? Is it too small? , maybe? Yeah, it's a bit
small. Yeah. One second. Let me try that. Ah, so yeah. Is this clearer? Let me
zoom in a bit more. A little better. Yeah. Yeah. So this was the one that got
everybody's attention.
[00:02:16] Andrew Liew Weida: Within the committee of mypodcast. So this is what you said ask yourself if you wanna continue reading.
Chat g BT bars or implementing open AI chat, g b D transformer, do you wanna
lead the path to innovation or vanish when the buzz fade away, yeah, of course.
And then you talk about, let me break this down for you.
[00:02:37] Andrew Liew Weida: Step one, step two, all the
[00:02:39] Denis Rothman: way. Yeah, I remember now.Yeah, I got it. Yes.
[00:02:42] Andrew Liew Weida: And this was the one thatgot a few of the listeners attention. And yeah, so before that I think let us
allow the audience to first know more about you before we comment on this post.
I think the first question is tell us your backstory.
[00:02:58] Andrew Liew Weida: How do you become an AI [00:03:00] practitioner from the day that youfinished school?
[00:03:02] Denis Rothman: Okay. I started my business asa student already when I was at the university. And the first business was
teaching languages. Okay. But very quickly, I had too many students. So I was
thinking of ways to automate the teaching.
[00:03:20] Denis Rothman: So I began writing softwarethat would create dialogues exactly exactly like chat, G b T, but with not,
it's not a probabilistic method with semantic method with rule bases and some
probabilities. So I developed some dialogue software and the L V M H, which
everyone knows is the largest luxury corporation in the world.
[00:03:45] Denis Rothman: But they were very interestedcuz they say we have many executives and they don't have time to, to move
around. So I installed this software in the laboratory where they could go sit
down and they could talk, they could communicate with a computer, and the dialogue was not. With artificial voice.
[00:04:05] Denis Rothman: It was with natural voices likewe do in chatbots today. And the dialogue was very very complex. It would
remember what the user would say we're in the 1980s. So people are saying, but
why? How's that possible? It was possible when people had big budgets. It's,
today, it's possible because it's mainstream, but it was al already possible in
jet fighters in the army, places like that, where there were a lot of budgets,
big budgets.
[00:04:35] Denis Rothman: You needed a lot of money. AndL V M H had big budgets so I could develop this software. And at the same time
Aeros Spatial, which is now Airbus had become Airbus the plane manufacturer.
Yeah. They were looking for artificial intelligence also because it's very
complex. Maintenance management is very complex on aircraft and battle ships.[00:05:00]
[00:05:00] Denis Rothman: So they were looking forsomething very in artificial intelligence to quickly solve maintenance
problems. So I developed that also at the same time in the 1980s. And as, as I
said, no one was very interested in this. At that time it was just beginning.
Of course, you had American universities making a lot of buzz and noise, but
but in the corpora corporations in the mostly aeros spatial, it was already
there.
[00:05:28] Denis Rothman: So I wasn't the only one. So Istarted there and that built very quickly my career because as soon as you get
customers for rockets or airplanes or battleships, yeah, you can enter all the
corporations you want. So that's how it started. And then after that, it's
just, , you can look at my profile.
[00:05:46] Denis Rothman: It's just a long string ofcorporations. Last one being like Disney or corporations like that. All my life
I had corporations with that had good budgets so that I could do practically
anything I wanted very early before became mainstream. That's why you see this post I'm used to implementing in
corporations.
[00:06:06] Denis Rothman: Yeah. Where it has to work. Youcan't just talk it, it works or you get out of the corporation very quickly.
Yes you can get kicked out of corporations as quick as you get in and you can
lose your reputation in one project. So it's it's like surfing on a big wave.
If you miss the wave, then you might die on a rock
[00:06:26] Andrew Liew Weida: Not, the interesting partis that every path has its own start that you mentioned about L V M H. Can you
do you still remember that moments where, how does Lv Hamish knock on your
doors? Or what was the conversation like that enabled that kind of use case?
Was it a translation English to France?
[00:06:44] Andrew Liew Weida: Or what was that story? Canyou elaborate about
[00:06:46] Denis Rothman: that? So how did both happen?The first one with Aeros spatial was I was at Sorbonne University, and there
was a politician that was a mathematician. He was a good mathematician and he [00:07:00] was very interested in this. So he, wetalked and he said I have a, I know a lot of people in the Aeros spatial field.
[00:07:07] Denis Rothman: Let's do a conference at theSeaburn. Let's see the reaction. And if I'll have them come, and if it's good,
then they'll contact you. I didn't know. So he organized a conference at the
Sobo. I made the conference. I didn't know that people were there. And then
after they contacted me, they say we were there and we're interested.
[00:07:26] Denis Rothman: So that's for the Aerosspatial. That led to Airbus and many companies that work around him that on the
other side, L v m Vmh had problems teaching training their executives that were
very busy. And the same thing happened. I was in Paris and I was talking to
consultants and they said, why don't you contact this guy?
[00:07:48] Denis Rothman: He knows he knows how to dothings that are very innovative. So I went to lvmh, I installed all this, and
what they did it's in an article, in fact, in on my LinkedIn [00:08:00] profile, there's an article that tellsthis story. And you e and what? L V M H, they even wrote the documentation and
they publicized it.
[00:08:08] Denis Rothman: So the documentation is one inone of my articles where I don't explain, they explain how they used it. And
then after that, once you have references like that, they just, people just
talk. They say go see him. It just, it's just a snowball effect that goes on
for years. I
[00:08:27] Andrew Liew Weida: see.
[00:08:27] Andrew Liew Weida: Interesting. And so tell memore as as you evolve from the 1980s to today, like how have you seen, like
what's your view on digital transformation, especially in terms of natural
language processing and large langu, large language models over the years? How
has it evolved as you practice in this space the first big step was in the
early 1990. where I implemented a very complex system for a company called afa.
It was part of the buyer [00:09:00] group,which is a big chemical company in the world, and AFA is a pretty big company
too. And I began using probabilistic algorithms. I began using Boltzmann
Boltzmann equations.
[00:09:13] Denis Rothman: I began using, oh a Frenchmathematician, chaos theory expert system with probabilistic outcomes. So very
quickly I saw that if you want to go faster, you can use probability. So there
were, so you have two paths. Yeah. You have a path, which is semantic. With
rule bases, expert systems controlled.
[00:09:36] Denis Rothman: That's 80% of the market. Eventoday, not 80% of the buzz. Is large language models, 80% of implementation of
automation is still classical expert system rule-based. But the 20% is
important too. But I saw that you could go quicker with probabilities. If you
take, you have big masses of data, then you can calculate.
[00:09:59] Denis Rothman: You can say [00:10:00] what comes after the a noun, what comesafter a noun? A verb. Okay. Western languages are very easy. You can't do
things like that, that easy in Chinese or the glutenin of languages, or you
don't need a verb. So that's what Western people don't understand. We use
verbs, but there are a lot of languages that don't need verbs, that you can
just put the words together and make sentences.
[00:10:23] Denis Rothman: So anyway, that followsprobabilities too. So I began introducing that and I used it a lot. I use his
techniques a lot and in fact, I've been doing AI all my life, but it didn't
interest anybody at that time. So in the 1990s I just delivered and I wouldn't
say anything. I just put the algorithm behind the user interface and if the per
person's happy he pays my bills, , and then the, and that's it.
[00:10:50] Denis Rothman: And then it went all the way upinto 2010, 2015. And then around 2015, Google came along and boosted artificial
[00:11:00] intelligence at once. They made itmainstream, but it had been going on for decades already in many fields, but all
of the, all of a sudden was mainstream. So once it was mainstream they got a
lot of budgets.
[00:11:12] Denis Rothman: And then you get open ai andthen you get to large language models, which are just statistics, basically.
You just take you take billions of sentences. Bill and you just say let's see
the probability that this sentence, this word will come after this word and
this word will come after that word and that, okay, so I'm going to go outside.
[00:11:32] Denis Rothman: Where can you go? Then you cango to your car, you can go to your garden, but it's not lightly. You're going
to say, I'm going to outer space. So you're gonna say, once I'm in the car,
where are you going? I go to the supermarket, to the movies. You're not gonna
say, I'm going into a jungle. So it's just statistics.
[00:11:49] Denis Rothman: Large language models arestatistics with a huge amount of parameters so that you can make a nice precise
vector space. And for those that don't [00:12:00]understand what vector spaces are, it's just like describing when you take a
picture, you can do it with not many pixels than more pixels than a million
pixels, than 10 million pixels and 12 million.
[00:12:11] Denis Rothman: The image will be more precise.So statistics will be more precise if you use more parameters. That's all
vector spaces are about. And then if you have huge ma big machines, then you
can create a lot of calculations and probabilities and add even more
parameters. But it still remains probabilities.
[00:12:30] Denis Rothman: It doesn't, there's nointelligence in there. There's no consciousness, there's nothing in there. It's
empty. If you want to fill it up, you're gonna have to add rule bases and
expert system. And this has already been this is already the case. I always did
that. I would do probabilistic and I would add rules also to make it seem more
intelligent.
[00:12:51] Denis Rothman: I would add expert in rules,the expert of a field the subject matter expert. Then I would an, I would enter
his rules so it would filter the [00:13:00]probabilities that come out. So if you say, I'm going outside, if the person is
specialized in gardens, it will always say you're going into the garden. Cuz I
know.
[00:13:10] Denis Rothman: He's a gardener, so I'll putthe rule in there. If you go outside, don't suggest the car, just suggest all
the probabilities with garden and garden tools and tractors if you have a farm,
but kick that out. So with the rule base, adding the booth together becomes
very powerful. Okay.
[00:13:27] Andrew Liew Weida: So what is interesting, Iwanted to ask is that, like you said, like back in the 1980s you started doing
it, and now there's already 2023 that has open ai.
[00:13:36] Andrew Liew Weida: Is it also has has the thestorage cost of hosting the data and the computing cost of the data actually
made these kind of work much easier? Or like how has that actually changed the
way that these kind of large language model is
[00:13:53] Denis Rothman: being done? Yeah. So yeah, thatthe budget is an interesting subject because people are always focusing on [00:14:00] the buzz theen, how intelligent it's goingto be.
[00:14:03] Denis Rothman: So I would say in the old days,and today also in corporations, there are big budgets. So if you're doing a
project, you can get a big budget to do your project, but that your project,
you're not gonna share it with the general public. It's gonna be confidential.
Yes. It's gonna be, or even classified so the general public won't have access
to it.
[00:14:24] Denis Rothman: But it's a big budget. Maybeit'll take a million dollars or $2 million over the, it's, these are very
expensive things. Yeah. So now you go to open ai, Microsoft, Google, they have
these big budgets and they're given the illusion to many people that, yeah,
sure. Why don't you train your budget. Sure. Give me give me 10 to million
dollars to buy a super computer, then you can give me another $50 million so I
can have 15 engineers for several years.
[00:14:54] Denis Rothman: And then at the end Microsoftsays, open it. I'm gonna put a billion dollars and open the eye. Which [00:15:00] means that now the money has gone fromlocal corporations to these huge distributors, but still people have to pay,
they're gonna have to pay for each token. They don't have to pay for everything
they use.
[00:15:15] Denis Rothman: So now people think it's funny,but they can't do anything without pain. They will, people will be able, will
have to pay for everything they do. Even if you go to a nice website like
hugging Face Yeah, transformers. It's a very nice thing. So you get the
illusion very quickly, oh yeah, I downloaded and I trained it, but it's not
gonna do much.
[00:15:35] Denis Rothman: You're not going to, you'regonna solve small problems. And if you have big problems to solve, then you
have to go to the paying part of hugging face and they're going to suggest, why
don't you use Amazon Web Service? Okay. But Amazon Web Services, it's limited
in the number of GPUs you can rent. And also Google AI Cloud is limited.
[00:15:55] Denis Rothman: If you're looking for GPUstoday, you're not gonna find many. Now, [00:16:00]when Open AI trains a transformer, how many GPUs do you think they use? 10,000.
Wow. 10,000 minimum. 10,000 GPUs. And people are saying, I have four GPUs.
Yeah, sure. I have a horse too. And with my horse, I'm going to go racing cars
so it's out of our reach already.
[00:16:20] Denis Rothman: It's in the hands ofcorporations, end of distribution corporation. But it's so expensive. It'll
always be that way, unless you can ha hire a hundred people. Nvidia has very
nice processors and you can rent very nice servers. But look at the cost. The
cost of renting it, then the cost of having two engineers.
[00:16:41] Denis Rothman: Then the cost of having 15engineers to control the quality and check the data, and then the access to the
data. Where you gonna get the data sets? Where do people think they're going to
get data sets? Download it from Kegel. Sure. That's but you're not gonna go, so
I'm going to go to a corporation [00:17:00]that sells airplanes and I'm gonna say I'm download your data set from Kaggle.
[00:17:04] Denis Rothman: You say it doesn't exist there,it don't exist in our company. And it's protected. It's not even on a cloud,
it's on a server. So if you want to use it, you're gonna sign 15 contracts for
confidentiality. Then we'll get the data set. And the data set might take you
two years Wow. To get it ready, because it's multiple source.
[00:17:23] Denis Rothman: This information is on thatdatabase. That information is another country. This one is there. So it's, I'm
saying it's been possible for a very long time, but it's very expensive. , but
it can be very profitable for those who know how to use it. Yeah. Okay. So
let's talk
[00:17:39] Andrew Liew Weida: about the future because asyou mentioned like lately probabilistic is beginning to gain more prominence
relative to the semantic rule base, which is currently still the dominant one.
[00:17:50] Andrew Liew Weida: As we move downward to thefuture, and you also mentioned about having experts and domain reasoning to tweak and make this transformer moreintelligence.