Sissie Hsiao, Vice President and General Manager, Google Assistant and Bard, Google Moderator: Jeremy Kahn, Senior Writer, FORTUNE; Co-chair, Fortune Brainstorm AI
Category
🤖
TechTranscript
00:00 - Hello, hi everybody, welcome.
00:02 Welcome to the 2023 Fortune Brainstorm AI Conference.
00:06 Great to see you all back here again in San Francisco.
00:10 This is the third year of Fortune's Brainstorm AI Conference.
00:14 Yes, we did get started before chat GPT, long before.
00:17 And many of you have been here with us from the beginning.
00:21 Artificial intelligence, as we all know,
00:23 is not a new concept.
00:24 In fact, it's been around since at least 1956
00:27 when Dartmouth hosted a summer research project
00:30 and conference.
00:30 And the concept was not all that different
00:32 from ours here today,
00:34 to gather some of the greatest minds in the field
00:37 and come to more or less of an agreement maybe
00:40 on a shared vision that computers can be made
00:43 to perform intelligent tasks.
00:44 I think we've established at least that much.
00:47 And now with the explosion, of course,
00:48 of open AI's chat GPT and other technologies.
00:52 Last November, generative AI has garnered the attention
00:55 of regulators as well, the general public
00:59 and businesses alike.
01:00 With AI's newfound attention and capabilities
01:03 comes a wave of both fear and excitement
01:07 on how it will be deployed in the coming years
01:10 and what effect it's going to have on our society as a whole.
01:14 AI is well on its way revolutionizing fields
01:17 like healthcare, entertainment, manufacturing, education,
01:20 and a whole lot more.
01:21 But with power comes great responsibility.
01:24 AI has the potential to perpetuate biases
01:28 and in the hands of bad actors, spread misinformation
01:32 and open the door to major security risks.
01:35 Many fear it's going to replace jobs.
01:38 Some have even argued that AI could even replace CEOs
01:43 and journalists.
01:45 Oh no.
01:46 With so much unknown and unregulated,
01:48 we look forward to insightful conversations
01:50 over these next two days as we examine new business cases
01:54 for AI and explore how it's already changing
01:57 the way we live, work, and do just about everything.
02:00 We have a stellar lineup of speakers,
02:02 but remember that this event was really created
02:04 for your participation and is meant to be
02:07 an interactive discussion among speakers and attendees.
02:10 The most important conversations are those among all of you.
02:15 As we navigate the innovations and the challenges
02:18 of implementing new technologies
02:19 and new ways of collaborating.
02:22 Before we start, some important acknowledgements.
02:24 I want to recognize the incredible work
02:27 of our Brainstorm AI co-chairs,
02:29 Fortune senior writer, Jeremy Kahn,
02:31 Fortune senior editor, Vern Kopitoff,
02:34 and co-founder and CEO of Afectiva, Dr. Rana Al-Khalioubi.
02:39 Thank you to our co-chairs.
02:42 (audience applauding)
02:46 We are also pleased to be joined
02:48 by additional Fortune colleagues
02:50 who are going to be moderating some of our discussions.
02:52 Alan Murray, CEO of Fortune,
02:54 Ellie Austin, Fortune's deputy editorial director
02:57 for Live Media, and Jeff John Roberts,
02:59 editor of Fortune Crypto.
03:01 Before we get started, I also want to extend
03:03 a tremendous thank you to our partners
03:05 for their contributions to this event.
03:07 Our presenting partner, Accenture,
03:10 our partners AARP, Amgen, Checkpoint,
03:14 Hancho, Salesforce, Workday,
03:18 and our supporting partner, Felicius.
03:21 Before we get into our sessions,
03:22 an important reminder about our conversations.
03:25 You probably get sick of us saying this,
03:27 but it's important.
03:28 What's said on the main stage
03:30 or during the strategy sessions
03:32 is on the record unless otherwise noted,
03:34 and what's said over a break or over dinner or a walk
03:38 is of course private unless agreed upon otherwise
03:41 by all parties.
03:42 Okay, let's get started.
03:44 As mentioned earlier, the release of OpenAI's Chat GPT
03:47 has brought worldwide attention
03:49 to the uses of generative AI, chatbots,
03:52 and the ability to release large language models
03:55 to the public.
03:56 A few months later, Google released
03:58 its conversational generative AI chatbot known as BARD.
04:02 Since then, Google merged the BARD teams
04:04 with the Google Assistant team
04:06 to create Assistant with BARD,
04:08 a new personal assistant chatbot powered by generative AI.
04:12 Google also unveiled a generative AI search experiment
04:15 that may alter how we all find information
04:18 and possibly disrupt digital business models
04:21 across industries.
04:22 Last Thursday, Google announced the release
04:25 of their newest model, Gemini,
04:26 which you all probably know all about,
04:28 and that is now going to power BARD.
04:31 Here to talk about the future of Gemini-powered BARD,
04:34 what's next for the chatbot,
04:36 please welcome General Manager
04:37 for Google Assistant and BARD, Sissy Hsiao.
04:40 She's going to be interviewed by Fortune senior writer
04:43 and Brainstorm AI co-chair, Jeremy Kan.
04:46 (audience applauding)
04:49 - Sissy, thanks for joining us,
04:56 and thank you all for joining us.
04:58 We're gonna go to questions from the audience,
05:00 hopefully towards the end of the session,
05:02 so please start thinking of your questions.
05:04 Sissy, big news last week with the unveiling of Gemini
05:09 and the announcement that Gemini
05:10 was now going to be powering BARD.
05:12 There was also a lot of disappointment, I think,
05:15 over the video of the demonstration
05:17 that came out with that announcement.
05:20 In hindsight, do you regret how that video was handled?
05:24 - Well, maybe let me start by saying
05:26 thanks for having me, Jeremy,
05:27 and thanks for having me for today.
05:30 Start there.
05:31 Very excited to be here to talk about
05:33 generative AI and BARD, of course.
05:36 I mean, let me start with the last part of your question.
05:38 The video is completely real.
05:40 All the prompts and the model responses are real.
05:43 We did shorten parts for brevity,
05:45 which we put in the video as informational
05:48 on the making of the video,
05:50 but we're also looking forward to releasing
05:52 the API in AI Studio this week
05:55 so that developers can play with it.
05:57 But that's really addressing the video.
05:59 I think the main headline story
06:02 is we've entered the Gemini era.
06:04 This is the best set of base models
06:06 in the sort of medium-sized range
06:08 and the large-sized range on a variety
06:11 of industry benchmarks that all researchers
06:13 benchmark their large language models off of.
06:15 And so I've been really excited to take that expression
06:18 of that power in those Gemini models
06:20 and put them in BARD,
06:22 which we saw with our side-by-side blind evaluations.
06:25 So BARD is the best, most preferred free chat bot now
06:30 in the market in blind evaluations with third-party raters.
06:34 - Great.
06:35 And what have been the most interesting use cases
06:37 you've seen so far with BARD,
06:38 or things that have really surprised you
06:40 that people are doing with it?
06:42 - I think what surprises me is just the breadth
06:46 of things that people do with these chat bots.
06:48 It's really hard to say it does just one thing
06:51 or the other thing, 'cause it does so many things.
06:53 So I'll give you an example.
06:55 For my personal use at my home life,
06:59 my daughter just got her ears pierced,
07:01 and I was looking for a particular type of earring
07:03 that she was looking for that wouldn't cause allergies.
07:06 I brainstormed with BARD about that.
07:08 BARD gave me some brands.
07:09 BARD also told me how to take care of your ears
07:12 to not get an infection.
07:13 That's a very personal use case
07:15 that was able to take that problem and solve it for me.
07:19 But at work, in productivity cases,
07:21 we have people using it for code.
07:24 I have a coworker whose English is a second language for him
07:28 and so he uses BARD to rewrite his emails,
07:30 to be more professional.
07:32 So really these AIs have just such a broad set of uses
07:37 because they can role play, they can generate content,
07:40 they can brainstorm with people.
07:42 So I'm really excited to see just the breadth
07:44 of things that people are trying
07:46 and also what really resonates with users in the world.
07:50 - And when we look ahead to Gemini,
07:52 what really excites you about what you're gonna be able
07:54 to do with BARD now powered by Gemini
07:56 that maybe you couldn't do six months ago?
07:59 - Yeah, well, I mean, first of all,
08:00 Gemini is just a stronger base model,
08:02 which means everything that comes out of it
08:04 is more high quality.
08:06 So we see it in text generation,
08:07 we see it in code generation,
08:09 we see it in instruction following.
08:11 So typically complex prompts have many,
08:14 many instructions embedded
08:15 and the model's able to follow those instructions.
08:18 But I think the most visually stunning
08:22 is, of course, multimodality.
08:24 This is where the model can ingest not just text,
08:27 but image and audio and video
08:30 and also communicate back out to the user
08:33 in not only text, but also in other means of communication
08:38 like video and image and audio.
08:41 A very simple example, I just used BARD to,
08:44 I went to a dinner, I had a menu that was like,
08:47 choose a course from the first, second, third course,
08:49 and then I had a wine list,
08:50 and I took a picture of the two menus side by side
08:53 and I said, choose me a dinner for tonight,
08:56 one course from each section
08:58 with a wine pairing to go with it,
09:00 and it actually did fabulously.
09:01 And so even something like that,
09:03 as simple as just taking a picture of something
09:06 and asking a question or for the bot to do something for you
09:09 will just become second nature
09:11 as these multimodal models become
09:14 as easy to access as your phone.
09:16 - Yeah, interesting.
09:18 And a lot of people are looking forward to a day
09:19 when these models, you can prompt them
09:22 to go out and do something for you,
09:24 to take an action on the internet on your behalf.
09:26 How far away is that kind of agency, do you think?
09:29 And will BARD be for Google,
09:31 the thing that carries out that,
09:33 your mission in the world for you?
09:35 - Yeah, I mean, getting things done for people
09:38 is definitely within reach of these agents,
09:42 or we call them agents
09:43 because they have this agentive capability.
09:45 So already BARD can summarize your emails, right?
09:48 We've connected BARD to Gmail, YouTube,
09:50 it can summarize YouTube videos for you,
09:52 you can ask questions.
09:54 So already this action ability is starting to emerge
09:57 when you connect different tools that exist in the world
10:00 into the large language model.
10:02 And foundationally, the reason why it's able to do this
10:04 is you're teaching the model to speak another language,
10:07 which is essentially code, right?
10:09 When it can speak APIs,
10:11 it can actually translate the user's request
10:14 and convert it into a set of API calls
10:17 with reasoning on top.
10:18 So I'll give you an example.
10:20 One thing I want BARD to be able to do for me personally
10:23 is book my entire kids' summer camps.
10:26 (audience cheers)
10:27 Woo, yeah, all the parents are happy.
10:29 Okay, I have two kids, they have different interests,
10:32 I need to find camps every week of the summer
10:34 that are within driving distance
10:36 and actually I can pick up and drop them off, right?
10:39 And so this is really hard, right?
10:41 So you have to think about all the reasoning
10:42 that goes into that.
10:43 And so I've gotten BARD to the point
10:45 where it's giving me three options per child per week,
10:48 but wouldn't it be awesome if I could just say,
10:50 "Please now book this."
10:52 We're almost there.
10:53 - Wow, how far away do we think?
10:55 - Things move really fast in this industry,
10:59 I can't say exactly when, but I think using tools
11:03 and not only using tools, but the logical reasoning on top
11:07 is very powerful and we're definitely working on it.
11:10 - One of the things that people have been concerned about
11:12 with all of these LM-powered chat interfaces
11:14 has been hallucination.
11:15 And I remember being here last year,
11:18 maybe some of you were also here last year,
11:19 and there were definitely people saying,
11:21 "Don't worry, hallucination is gonna be solved
11:22 "within nine months, it's gonna be,
11:24 "come back next year, it'll be solved."
11:25 It's not solved really, but how are we doing
11:28 on making progress on hallucination?
11:29 And do you think that this is a solvable problem?
11:32 - Look, so hallucinations are inherent
11:36 to all large language models.
11:37 Now we even know this word in the lingo of this industry,
11:40 we all are aware of this, and before we didn't
11:43 use this word very much.
11:44 It's two-pronged approach.
11:46 One is, can you actually just improve the issue, right?
11:50 Can you make the bot hallucinate less?
11:54 The answer is absolutely yes.
11:56 And that takes hard work, fine-tuning, evaluation,
11:59 and we've seen the hallucination rate decrease in BARD
12:02 over the last eight months.
12:03 We've been working on it really hard,
12:05 and it's definitely improving.
12:06 And I would also say the smarter the base model,
12:09 the more it tends to understand how to use search
12:13 as a tool to really reduce its hallucinations.
12:16 So that's like sort of one step.
12:17 The other step is we're first to market
12:19 to launch something we call DoubleCheck.
12:22 In the industry, it's called an entailment model,
12:24 but really what it is is checking all the sentences
12:26 that come out of the bot with search.
12:29 And if you think about it, if you're writing an essay
12:31 or doing something important, as a human,
12:34 you're gonna be doing a lot of searching
12:36 to corroborate your facts, right?
12:37 You're gonna be reading all of the different,
12:39 maybe dissenting views, and really deciding for yourself
12:42 what is the right way to write
12:44 any particular piece of content.
12:46 So we put DoubleCheck into BARD to help people
12:49 really quickly get access to search corroboration
12:53 on top of the model's output.
12:54 And that's yet another way that if you're doing something
12:57 like really, really important that it must be factual,
13:00 you can use DoubleCheck as another tool
13:02 to help you really understand what BARD is saying.
13:06 - Interesting, and do you think that BARD and search,
13:08 which right now are very distinct products,
13:10 will they remain distinct, or are we going to see
13:11 increasing kind of merger between the two?
13:13 - You know, I think of search and SGE,
13:17 which is the application of large language models
13:19 inside search, and BARD as really complementary.
13:23 AI is a raw technology that has application
13:27 in lots of products, not just search
13:29 and standalone chatbots, but I'm sure many of you
13:32 in the room are thinking about how can this transform
13:33 like my business operations and the way we do work.
13:36 And so I think that the way I think about it very simply
13:39 is you go to search because you mostly
13:41 are wanting information, right?
13:42 And I think SGE does a fantastic job
13:45 sort of summarizing what the web is really saying
13:48 about your question, whereas BARD is like
13:51 you are directly tunneled into the AI, right?
13:53 And it has this full amplitude of expression.
13:55 You can have it write poems,
13:56 you can have it critique your code,
13:59 you can brainstorm ideas for a birthday party with the bot,
14:02 you can have a conversation just for fun.
14:04 And so it's just kind of a very different expression
14:08 of what underlying is basically
14:11 a similar base model technology.
14:13 - Great, I wanna try to get questions.
14:15 Does anyone have a question from the audience?
14:17 If we can get a mic to the gentleman down front
14:21 here at the first table, please.
14:24 And if you could please state your name
14:26 and where you're from.
14:27 - Hi, I'm Edward Hall.
14:30 I teach AI at NYU and I run Intellibus
14:33 and I'm behind the big parser data commons project.
14:36 So question is how does this transform
14:40 Google's business model?
14:41 OpenAI's business model is charged subscriptions
14:43 to consumers.
14:44 And so while of course the internet is full of advertising,
14:47 how does a consumer get clean information out of this
14:51 and what's the transformation of the model, business model?
14:54 - You know, really we're really focused in Bard
14:58 on making the AI very, very helpful for people.
15:01 You know, monetization is something that, you know,
15:03 we're always looking at,
15:06 but it's not really our focus right now.
15:08 It's really about like how do you make this technology
15:10 really great and help people with their everyday tasks.
15:14 - And other questions.
15:16 Who else has a question?
15:18 Raise your hand.
15:19 Oh, the gentleman here at this table.
15:21 Wait till the mic gets you.
15:22 There you go.
15:23 - Yeah, hi, this is Siddhartha Agarwal with Freshworks.
15:25 And you know, my question is around security
15:28 for the models because people are doing prompt hijacking
15:32 and being able to understand what's coming out.
15:34 So is there gonna be the need for firewalls
15:36 for these models?
15:40 - Well, I mean, security has like different facets, right?
15:44 One is, for example, if you have specific data
15:47 that you're grounding the model against,
15:49 you don't wanna leak that data, right?
15:51 So you need to make sure that any data
15:55 that's used to flavor your model
15:57 in any particular way is protected.
16:00 There's also ways, for example, when you use tools
16:04 to make sure that the chatbot is operating tools
16:08 in a sandbox, right?
16:09 So there's different layers of security
16:11 that we add to avoid prompt hijacking,
16:14 avoid leaking grounding data or training data,
16:17 or make sure that its execution is safe.
16:21 So in BARD, we do a lot of red teaming.
16:23 We have adversarial testing to make sure
16:26 that we're constantly looking at the model
16:29 and making sure that it has different layers of protection
16:31 around all these potential exploits that might occur.
16:35 - Interesting.
16:35 Next question, just here.
16:38 - I'm gonna stand up on a tablecloth.
16:40 - Oh, dear. - Hi, I'm Ann Michael,
16:41 of the American Institute of Physics Publishing.
16:43 And so I wanna go back to what you were saying
16:46 about the distinction between search and the use of an AI,
16:49 because fundamentally, we search
16:52 because we had to break down what we wanted to do
16:54 into these obscure pieces to find something.
16:57 But if we could have just asked,
16:59 we never would have searched.
17:01 We would have just gone right to, you know,
17:03 it's almost, it's kind of a, it's a choppy way.
17:07 And I guess when I'm thinking back
17:08 about the gentleman's question about business model,
17:11 can you see a time when we don't search, we ask,
17:16 and then how do you get ad revenue from that?
17:19 - Like I said earlier, I think, you know,
17:23 there's the two parts of the question.
17:24 One is as the AI gets smarter,
17:26 can it take tasks off your plate?
17:28 I think our endeavor is to make that yes, right?
17:31 Because we think that AIs have such incredible aptitude,
17:35 like solve my summer camp problem, right?
17:36 That's a very complicated problem, right?
17:38 With a lot of steps to that.
17:40 In terms of, you know, is it ads,
17:42 is other forms of monetization?
17:44 It's just very early and it's not something
17:46 that like I could say within--
17:47 - Do you think we're gonna see,
17:48 are we gonna see with BARD a kind of store?
17:50 I mean, OpenAI has talked about trying to allow people
17:53 to create their own models
17:54 and maybe have some kind of a kind of app store
17:56 for a model store.
17:57 Are you thinking about that for BARD?
18:00 - You know, I think it's great
18:02 that there's so much innovation, right?
18:03 Like customizing an LLM into a particular type of behavior
18:08 is something that I think some people wanna do,
18:11 especially power users.
18:12 So I'm interested in how people wanna use that.
18:14 You know, right now we're really focused
18:16 on making BARD really, really helpful.
18:19 And why I drew that distinction is because
18:23 BARD can play act many different flavors of behavior, right?
18:27 Because you can just prompt it
18:28 into being an interior designer or a statistician
18:31 and you can actually prompt your way
18:33 to many different flavors of it, if you will.
18:36 So I think, you know, whether you need to really split
18:39 that bot into many little sub bots
18:41 or whether the main bot is really helpful at many things,
18:44 like these are just different models
18:46 of expressing the core technology.
18:48 And I'm very interested in how this will work for people.
18:52 - Well, Sissy, we're out of time.
18:53 Thank you so much for joining us
18:55 and thank you all for listening.
18:57 - All right, thank you so much.
18:58 (audience applauding)
19:01 [BLANK_AUDIO]