WEBVTT 00:00.720 --> 00:03.160 Welcome back everyone . Our final panel 00:03.170 --> 00:05.789 for today is the Emerging Technologies 00:05.800 --> 00:09.539 panel moderated by Ramesh Ramesh 00:09.550 --> 00:11.789 Menon . Mr Minon is the Chief 00:11.800 --> 00:14.022 technology officer in the Office of the 00:14.022 --> 00:17.129 Cio for Dia Mr Minon . I will turn it 00:17.139 --> 00:18.250 over to you , sir , 00:22.719 --> 00:25.620 to the sky . What 00:28.500 --> 00:29.770 we will stand 00:32.009 --> 00:35.849 faces together . Let's 00:35.860 --> 00:36.180 go 00:42.060 --> 00:45.770 understand . Good 00:45.779 --> 00:49.080 afternoon . Welcome . It's the last day 00:49.090 --> 00:51.312 here at Dudas . Hope you all had a good 00:51.312 --> 00:53.569 time , had an exciting session 00:54.119 --> 00:56.930 interacting with your colleagues as we 00:56.939 --> 01:00.419 are here final day in Portland and I'm 01:00.430 --> 01:02.652 glad it didn't rain . Hopefully you got 01:02.652 --> 01:06.139 to check out the city . I'm excited to 01:07.000 --> 01:10.779 have a very inspiring and 01:10.790 --> 01:12.957 intellectually simulating conversation 01:13.400 --> 01:15.622 with some of the brightest minds within 01:15.622 --> 01:19.449 dod . I would let each of our panel 01:19.459 --> 01:22.470 member introduce themselves what they 01:22.480 --> 01:25.489 do where they come from and 01:25.589 --> 01:29.269 why is now a very important time 01:29.900 --> 01:32.011 in the history of humanity and for us 01:32.011 --> 01:34.290 in national security to accelerate the 01:34.300 --> 01:36.330 speed and scale of innovation to 01:36.339 --> 01:38.450 achieve the capabilities and outcomes 01:38.450 --> 01:40.617 we need as a nation . I'm also pleased 01:40.617 --> 01:43.059 to announce that last week , General 01:43.069 --> 01:45.660 Barrier signed off on D ISA I strategy . 01:45.889 --> 01:48.379 It's a work by a lot of people , 01:48.389 --> 01:50.860 including our J two Q coms , our intel 01:50.870 --> 01:53.510 service centers , our leaders across 01:53.519 --> 01:56.269 the community in creating a holistic 01:56.290 --> 01:58.457 integrated strategy that will move our 01:58.457 --> 02:01.919 nation forward with that . Doctor 02:01.930 --> 02:04.620 Fisher . Can you please introduce 02:04.629 --> 02:06.907 yourself ? She's a director from DARPA . 02:06.907 --> 02:08.962 And the way I remember , I asked her 02:08.962 --> 02:11.073 when we were initially talking , what 02:11.073 --> 02:13.240 do you do when she told ? Oh , my team 02:13.240 --> 02:15.018 invented the internet . So that 02:15.018 --> 02:16.740 reminded me . OK , that's good 02:16.740 --> 02:18.740 introduction . So , Doctor Fisher , 02:18.740 --> 02:20.851 please . Ok . So I'm not the director 02:20.851 --> 02:22.796 of DARPA . I'm the Director of the 02:22.796 --> 02:24.962 Information Innovation Office at DARPA 02:24.962 --> 02:27.073 Stephanie Tompkins is the director of 02:27.073 --> 02:29.296 DARPA . So Darpa's mission is detecting 02:29.296 --> 02:31.240 uh and CRE preventing and creating 02:31.240 --> 02:33.296 strategic surprise . So looking over 02:33.296 --> 02:35.185 the horizon for opportunities and 02:35.185 --> 02:37.296 threats that might be coming , uh The 02:37.296 --> 02:39.518 Information Innovation Office does that 02:39.518 --> 02:39.509 in the space of information and 02:39.520 --> 02:42.169 computer science broadly construed . Uh 02:42.179 --> 02:44.800 So we have four thrust areas that we 02:44.809 --> 02:46.809 use to organize our activities . So 02:46.809 --> 02:49.087 they're descriptive , not proscriptive . 02:49.087 --> 02:50.753 So one of those is proficient 02:50.753 --> 02:52.920 artificial intelligence and you know , 02:52.920 --> 02:52.419 nothing really is happening in 02:52.429 --> 02:54.373 proficient artificial intelligence 02:54.373 --> 02:56.318 these days . Um So we use the word 02:56.318 --> 02:59.595 proficient to mean uh trust worthy , 02:59.684 --> 03:02.244 explainable all those kind of , you 03:02.255 --> 03:04.675 know , extra uh discreet adjectives to 03:04.684 --> 03:06.764 mean , you know , generally good uh 03:06.774 --> 03:09.395 doing what you wanted to do , et cetera . 03:09.404 --> 03:11.515 Uh a catchphrase we often use is that 03:11.515 --> 03:13.626 you would be willing to trust it with 03:13.626 --> 03:16.089 your life and not be stupid to do So um 03:16.160 --> 03:19.240 a second thrust area is information 03:19.250 --> 03:22.350 integrity . So that's missing 03:22.360 --> 03:25.179 disinformation . How do we monitor that ? 03:25.190 --> 03:27.190 How do we respond to that ? Because 03:27.190 --> 03:29.023 obviously our nation state level 03:29.023 --> 03:30.968 adversaries are spending a massive 03:30.968 --> 03:32.912 amount of money and energy in that 03:32.912 --> 03:35.369 space , the integrity of our scientific 03:35.380 --> 03:37.436 ecosystems , our supply chains , our 03:37.436 --> 03:39.658 financial system , all of that falls in 03:39.658 --> 03:42.184 that space , the exquisite surveillance 03:42.195 --> 03:45.244 systems that are possible with the um 03:45.354 --> 03:47.725 digital dust that is left behind when 03:47.735 --> 03:51.095 we interact with the modern smartphone 03:51.104 --> 03:53.904 and web ecosystem . How all of that the 03:53.914 --> 03:56.081 the digital dust we leave behind is an 03:56.081 --> 03:58.247 exquisite surveillance device . All of 03:58.247 --> 04:00.303 that is in the information integrity 04:00.303 --> 04:02.358 part of our portfolio . Then we have 04:02.358 --> 04:04.025 offensive and defensive cyber 04:04.025 --> 04:06.081 operations . How do we manage that ? 04:06.081 --> 04:08.081 And then the last part is resilient 04:08.081 --> 04:10.247 systems , resilient um and um flexible 04:10.247 --> 04:12.414 systems . And you can think of that as 04:12.414 --> 04:14.414 how do we reduce our digital attack 04:14.414 --> 04:16.247 surface so that we uh if in that 04:16.247 --> 04:18.470 offensive and defensive cyber operation 04:18.470 --> 04:20.581 space , how do we make ourselves less 04:20.581 --> 04:22.747 vulnerable in that space ? And I think 04:22.747 --> 04:24.858 with that , I will stop . Thank you , 04:24.858 --> 04:27.081 Doctor Fisher . Doctor Shanay is a very 04:27.081 --> 04:29.192 senior leader in the US Department of 04:29.192 --> 04:31.414 Defense . I had the opportunity to work 04:31.414 --> 04:33.525 with him . Probably some of you heard 04:33.525 --> 04:35.692 me speak at the dia booth yesterday as 04:35.692 --> 04:37.858 we were talking about the neuromorphic 04:37.858 --> 04:37.739 A I chip . It's one of the transit . We 04:37.750 --> 04:39.917 are one of the transition partners for 04:39.917 --> 04:42.380 dod which was fun led by DARPA and Rne . 04:42.649 --> 04:45.190 So in his role as principal director at 04:45.200 --> 04:47.367 us , Dr and E , he's leading the micro 04:47.367 --> 04:49.422 electronics and the whole story with 04:49.422 --> 04:51.367 chips . And how do we compete with 04:51.367 --> 04:53.089 China in this era of strategic 04:53.089 --> 04:55.089 competition ? So I would let Doctor 04:55.089 --> 04:57.200 Shenoy introduce what his office does 04:57.200 --> 04:59.422 and what his priorities are . Yeah , so 04:59.422 --> 05:01.367 I'm within the office of the under 05:01.367 --> 05:00.880 Secretary of Defense Research and 05:00.890 --> 05:02.946 Engineering , principal Director for 05:02.946 --> 05:05.700 Micronics . Um and there are 14 05:05.709 --> 05:07.431 principal directors in various 05:07.431 --> 05:09.570 technical areas , micro electronics , 05:09.579 --> 05:11.950 as you obviously know , underpins a lot 05:11.959 --> 05:13.903 of the critical technology areas , 05:13.903 --> 05:16.119 whether it's A I autonomy , robotics , 05:16.170 --> 05:18.640 five G , six G communications radar and 05:18.649 --> 05:20.816 so on and so forth . Micro electronics 05:20.816 --> 05:23.070 is critical to ensure the capabilities 05:23.079 --> 05:25.579 that the defense department has . Uh So 05:25.589 --> 05:27.811 within micro electronics , uh we've got 05:27.811 --> 05:30.149 a couple of efforts . One is new , it's 05:30.160 --> 05:32.899 called the Microelectronics Commons and 05:32.910 --> 05:35.109 this was recently launched using the 05:35.119 --> 05:37.500 Chips and Science Act funding and 05:37.510 --> 05:39.566 that's 400 million per year over the 05:39.566 --> 05:41.621 next five years . So it's $2 billion 05:41.621 --> 05:44.420 for chips rel related to DARPA uh sorry 05:44.429 --> 05:47.320 dod S needs and requirements . And then 05:47.329 --> 05:49.600 the um you know , second program is 05:49.609 --> 05:51.929 structured and microphonic which is to 05:51.940 --> 05:55.760 ensure that dod has access to leading 05:55.769 --> 05:58.209 edge capabilities in the US across the 05:58.220 --> 06:01.010 cycle of design , manufacturing , 06:01.019 --> 06:03.769 assembly , packaging and testing . So a 06:03.779 --> 06:05.946 lot of that ecosystem is offshore . As 06:05.946 --> 06:08.057 you know , it's a global supply chain 06:08.057 --> 06:10.168 it's a very complex supply chain . We 06:10.168 --> 06:11.946 still rely to a large extent on 06:11.946 --> 06:13.835 offshore suppliers , whether it's 06:13.835 --> 06:16.112 assembly packaging , testing and so on , 06:16.190 --> 06:18.246 including countries like China . And 06:18.246 --> 06:20.412 then so how do we reduce that reliance 06:20.412 --> 06:22.529 on our offshore suppliers ? We have 06:22.540 --> 06:24.373 through this program that I just 06:24.373 --> 06:26.540 mentioned , set up an ecosystem if you 06:26.540 --> 06:29.730 will for onshore suppliers to provide 06:29.739 --> 06:32.320 that access , access to secure design , 06:32.619 --> 06:35.250 access to secure manufacturing through 06:35.260 --> 06:38.040 onshore foundries , access to assembly 06:38.049 --> 06:40.049 packaging and testing again , using 06:40.049 --> 06:42.160 onshore suppliers . So we've actually 06:42.279 --> 06:45.720 de develop this access onshore , which 06:45.730 --> 06:47.786 is a game changer for the department 06:47.786 --> 06:49.730 because uh for literally decades , 06:49.730 --> 06:51.786 we've relied on offshore suppliers . 06:51.786 --> 06:54.008 I'll be happy to share more about it as 06:54.008 --> 06:56.230 we go along . Yeah . Thank you , Doctor 06:56.230 --> 06:58.839 Shay and then doctor Bill Sterle , good 06:58.850 --> 07:01.072 friend . We work very closely on day to 07:01.072 --> 07:03.128 day based on a lot of things . He is 07:03.128 --> 07:05.294 from office of Secretary of Defense uh 07:05.294 --> 07:07.517 CD A O . So , Doctor Sterle , why don't 07:07.517 --> 07:09.739 you introduce yourself and tell what CD 07:09.769 --> 07:11.991 O is doing and what's happening in that 07:11.991 --> 07:14.047 area in Os D ? Sure , thank you very 07:14.047 --> 07:15.769 much . Yeah . So I'm the Chief 07:15.769 --> 07:17.880 technology Officer in the , the , the 07:17.880 --> 07:20.047 Chief Digital and A I Office uh within 07:20.047 --> 07:22.158 the Department of Defense . And uh my 07:22.158 --> 07:24.760 role in that organization is to along 07:24.769 --> 07:26.880 with my colleagues in my office is to 07:26.880 --> 07:28.991 develop sort of a reference vision or 07:28.991 --> 07:30.880 architecture that helps chart the 07:30.880 --> 07:34.829 course for how A I major initiatives to 07:34.839 --> 07:37.260 improve that with education but also 07:37.269 --> 07:39.047 new new career paths within the 07:39.047 --> 07:41.213 department . Now , those three sort of 07:41.213 --> 07:43.529 foundational activities support two 07:43.739 --> 07:45.961 demonstrations of the effects of that . 07:45.961 --> 07:48.410 One is J AC two , we're responsible for 07:48.420 --> 07:50.642 the data integration layer for J AC two 07:50.642 --> 07:52.698 joint all domain command and control 07:52.698 --> 07:54.619 and also for the guide series of 07:54.630 --> 07:56.852 experiments in which we demonstrate the 07:56.859 --> 07:58.915 power of data analytics and A I in a 07:58.915 --> 08:01.026 war fighting context and leave behind 08:01.026 --> 08:03.248 capabilities to move that forward . And 08:03.248 --> 08:06.299 the third , the fifth one is um metrics 08:06.309 --> 08:09.000 for uh the deputy to monitor how her 08:09.010 --> 08:11.121 direct reports are accomplishing what 08:11.121 --> 08:13.177 they need to do on their uh on their 08:13.177 --> 08:15.177 road maps . Now , I said five , but 08:15.177 --> 08:17.343 actually , uh in the last year , we've 08:17.343 --> 08:19.621 added plus one which is generative A I . 08:19.621 --> 08:21.677 So uh in the course of answering our 08:21.677 --> 08:21.549 questions , I'm sure I'll get to it , 08:21.559 --> 08:23.615 but uh we have an effort called task 08:23.615 --> 08:26.230 force Lima which is bullish , I will 08:26.239 --> 08:29.049 say on A I and generative A I , but we 08:29.059 --> 08:31.115 want to do it responsibly . Uh And I 08:31.115 --> 08:33.281 hopefully we'll talk more about that . 08:33.281 --> 08:36.349 Thank you Bill . Since you mentioned CD 08:36.369 --> 08:38.147 OS mission is to accelerate the 08:38.147 --> 08:40.789 adoption of data analytics and A I 08:40.799 --> 08:43.469 across dod , what do you see as the 08:43.479 --> 08:46.739 major challenges to adoption and what 08:46.750 --> 08:48.739 are you doing to address them ? Uh 08:48.750 --> 08:50.972 Great question . So some of them I I've 08:50.972 --> 08:53.083 hit on a little bit , I would say one 08:53.083 --> 08:55.139 of the major challenges is access to 08:55.139 --> 08:57.083 data . Uh When we're talking about 08:57.083 --> 08:59.361 addressing those strategic initiatives , 08:59.361 --> 09:01.417 we've got this uh construct known as 09:01.417 --> 09:04.400 the A I HI hierarchy of needs . Uh And 09:04.659 --> 09:06.603 it's sort of sort of like the food 09:06.603 --> 09:08.659 pyramid . Um but it's for A I and at 09:08.659 --> 09:10.603 the very bottom is data . You need 09:10.603 --> 09:12.548 quality data , vegetables . Yeah , 09:12.548 --> 09:14.826 exactly . You've got their data tables , 09:14.826 --> 09:16.950 not veggie tables , but basically you 09:16.960 --> 09:19.071 need to get your data right or you're 09:19.071 --> 09:20.960 not going to get any capability , 09:20.960 --> 09:22.904 whether it's analytics or A I . So 09:22.904 --> 09:25.016 that's our one of our major uh thrust 09:25.016 --> 09:28.200 areas because um that is like literally 09:28.210 --> 09:30.432 the number one impediment to leveraging 09:30.432 --> 09:32.654 A I . Other impediments have to do with 09:32.654 --> 09:35.630 uh misunderstanding about A I uh lack 09:35.640 --> 09:38.239 of awareness . Um A I we often have to 09:38.250 --> 09:40.630 say is , is not a monolithic thing . 09:40.640 --> 09:43.250 It's not here as you might see in a 09:43.260 --> 09:45.580 Hollywood movie . It's a series of 09:45.590 --> 09:47.679 tools , algorithms really that learn 09:47.690 --> 09:49.634 patterns from data and then can be 09:49.634 --> 09:51.919 leveraged to yield insight , generative 09:51.929 --> 09:53.818 A I is a type of A I that is very 09:53.818 --> 09:55.929 powerful , but there's a lot that can 09:55.929 --> 09:58.096 be done with so called narrow A I . So 09:58.096 --> 10:00.318 helping to educate folks on that , make 10:00.318 --> 10:02.373 us better consumers and also help us 10:02.373 --> 10:04.540 communicate with vendors about what it 10:04.540 --> 10:06.762 is we really need . Um A third key area 10:06.762 --> 10:09.169 is uh in the area of policy actually , 10:09.229 --> 10:11.590 um the policy that we have currently 10:11.599 --> 10:13.599 and I'm using broad strokes here uh 10:13.599 --> 10:16.940 relates to data uh sharing of that and 10:16.950 --> 10:19.006 we really need to do a better job of 10:19.006 --> 10:21.283 understanding what it means for models . 10:21.283 --> 10:23.283 Uh you know , the policy associated 10:23.283 --> 10:25.172 with classification of data . For 10:25.172 --> 10:27.283 instance , we don't really understand 10:27.283 --> 10:27.090 what it means for a model that , that 10:27.099 --> 10:29.321 is trained on that . And you and I have 10:29.321 --> 10:31.377 talked about that several times . So 10:31.377 --> 10:33.488 those are some of the key impediments 10:33.488 --> 10:33.219 that we're working on . Thank you , 10:33.229 --> 10:36.270 Bill and Doctor Shanay . I mean , you 10:36.280 --> 10:38.929 are a busy person . You are in the news 10:38.940 --> 10:41.130 with Chips Act , everybody's tracking 10:41.140 --> 10:43.340 what you're doing . So what are dod 10:43.349 --> 10:45.849 strategic technology challenges to 10:45.859 --> 10:48.349 ensure military superiority and 10:48.359 --> 10:50.890 warfighter readiness ? Yeah . So you 10:50.900 --> 10:54.049 know , when we think about um how uh 10:54.059 --> 10:56.119 wars are being fought today , uh 10:56.130 --> 10:58.297 there's been a significant shift , you 10:58.297 --> 11:00.780 know , this supply chain that I alluded 11:00.789 --> 11:03.179 to in micro electronics . Um some of 11:03.190 --> 11:05.412 the v vulnerabilities and weaknesses of 11:05.412 --> 11:07.468 the supply chain were exposed during 11:07.468 --> 11:09.357 the pandemic . And so that's been 11:09.357 --> 11:11.412 exacerbated for the dod to be in the 11:11.412 --> 11:13.634 sense that the access that's needed for 11:13.634 --> 11:15.634 our weapon systems and platforms uh 11:15.634 --> 11:17.968 that's not readily available . And then , 11:17.968 --> 11:20.320 so we have to think about ensuring that 11:20.330 --> 11:23.669 there is an onshore source for all of 11:23.679 --> 11:27.000 the chips that we need for the dod and 11:27.010 --> 11:29.010 obviously the internal community as 11:29.010 --> 11:31.232 well . And then , so that's what we are 11:31.232 --> 11:33.121 facilitating , that's what we are 11:33.121 --> 11:35.288 enabling through the chips and science 11:35.288 --> 11:37.510 act funding onshore suppliers for these 11:37.510 --> 11:39.732 chips that are critical for the dod and 11:39.732 --> 11:41.677 the intel community . Uh Now , the 11:41.677 --> 11:43.899 other challenge that we have is when we 11:43.899 --> 11:45.899 think about whether it's a swarm of 11:45.899 --> 11:47.954 drones type of strategy , you know , 11:47.954 --> 11:50.066 being able to support the microphonic 11:50.066 --> 11:52.066 that's needed in those platforms is 11:52.066 --> 11:54.232 critical as well . So timely insertion 11:54.232 --> 11:56.288 of technology , timely adaptation of 11:56.288 --> 11:58.510 technology from the commercial sector , 11:58.510 --> 12:00.566 you know , that's a huge challenge , 12:00.566 --> 11:59.869 you could request qualification , 11:59.880 --> 12:02.047 certification and so on and so forth . 12:02.047 --> 12:03.658 So that timely transition of 12:03.658 --> 12:05.658 technologies is one of the critical 12:05.658 --> 12:07.991 challenges that we're trying to address . 12:07.991 --> 12:07.390 And the Micro Electron Commons 12:07.400 --> 12:09.809 initiative that I alluded to is helping 12:09.820 --> 12:11.709 address that , you know , pulling 12:11.709 --> 12:13.931 research out of the Univ start up small 12:13.931 --> 12:16.209 companies into prototyping and the fab . 12:16.209 --> 12:17.876 So the foundries where do the 12:17.876 --> 12:19.764 manufacturing and then eventually 12:19.764 --> 12:21.487 ensuring that there is further 12:21.487 --> 12:23.598 maturation of this technology , scale 12:23.598 --> 12:25.764 up of those technologies to help level 12:25.764 --> 12:27.876 up the systems and platforms that are 12:27.876 --> 12:30.098 critical for the dod . So tech adaption 12:30.098 --> 12:29.559 is , is really one of the biggest 12:29.570 --> 12:31.403 challenges that we have . We are 12:31.403 --> 12:33.403 certainly addressing it through the 12:33.403 --> 12:35.626 programs that we are supporting . Thank 12:35.626 --> 12:37.570 you , Doctor Shenoy . I absolutely 12:37.570 --> 12:39.570 concur with you that supply chain , 12:39.570 --> 12:41.570 supply chain assurance and ensuring 12:41.570 --> 12:43.681 economic prosperity is very important 12:43.681 --> 12:42.739 for us as a nation . And what you're 12:42.750 --> 12:46.030 doing is foundational for us for couple 12:46.039 --> 12:48.261 of decades coming going forward . Thank 12:48.261 --> 12:51.859 you . So , Doctor Fisher discerning 12:51.869 --> 12:53.830 manipulated information from real 12:53.840 --> 12:56.710 information and intent behind is likely 12:56.719 --> 12:58.497 to become a growing concern for 12:58.497 --> 13:00.750 intelligence analysts . How is DARPA 13:01.580 --> 13:03.469 tackling this issue ? And can you 13:03.469 --> 13:06.250 discuss the most promising technologies 13:06.270 --> 13:08.590 you see coming out of industry and your 13:08.599 --> 13:10.859 own efforts at DARPA ? Sure . Yes , 13:10.869 --> 13:13.091 we've been talking about generative A I 13:13.091 --> 13:15.313 and one of the things that generative A 13:15.313 --> 13:17.480 I is really good at is producing uh at 13:17.480 --> 13:20.849 the moment audio and uh still images 13:20.859 --> 13:24.140 that look very convincing . Um I've 13:24.150 --> 13:26.789 actually had two friends receive uh 13:26.799 --> 13:29.539 phone calls that sounded just like one 13:29.549 --> 13:31.605 of their loved ones claiming to have 13:31.605 --> 13:33.710 been kidnapped , which is just 13:33.719 --> 13:35.941 terrifying , right ? And when you're in 13:35.941 --> 13:37.941 that state of mind , like you , you 13:37.941 --> 13:40.163 think you've just received a phone call 13:40.163 --> 13:42.386 from , you know , your daughter or your 13:42.386 --> 13:41.650 son and they're saying they've been 13:41.659 --> 13:43.548 kidnapped and police send money , 13:43.548 --> 13:45.603 you're not in a good state of mind . 13:45.919 --> 13:47.729 And so that technology is , is 13:47.739 --> 13:50.409 terrifying in that regard . Um And so 13:50.419 --> 13:52.363 like to , to counter this threat , 13:52.363 --> 13:54.252 there's actually a whole bunch of 13:54.252 --> 13:56.419 things we need to do . Some of this is 13:56.419 --> 13:58.586 technological , some of it is policy , 13:58.586 --> 14:00.808 some of it is regulation . So DARPA has 14:00.808 --> 14:00.229 actually been working on this thread of 14:00.239 --> 14:03.419 detecting manipulated media for uh 89 14:03.429 --> 14:05.429 years before deep fakes were even a 14:05.429 --> 14:07.540 thing with the metaphor program . And 14:07.540 --> 14:09.429 now the SEMA four program . So we 14:09.429 --> 14:11.651 currently have detectors that are quite 14:11.651 --> 14:13.762 good at detecting manipulated audio , 14:13.762 --> 14:15.762 manipulated um images , manipulated 14:15.762 --> 14:17.985 video as well . Um that are looking for 14:17.985 --> 14:21.150 inconsistencies in ways that you can um 14:21.159 --> 14:23.710 the tools can show even a layperson 14:23.719 --> 14:25.830 where there are inconsistencies where 14:25.830 --> 14:27.941 their things are manipulated . So the 14:27.941 --> 14:29.886 layperson doesn't have to trust an 14:29.886 --> 14:31.997 expert but can look and see that . Oh 14:31.997 --> 14:33.663 yes , this is manipulated the 14:33.663 --> 14:35.830 technology for in video . It turns out 14:35.830 --> 14:38.052 when people speak , for example , there 14:38.052 --> 14:40.052 are micro expressions in their face 14:40.052 --> 14:42.163 that like when I say the word hello , 14:42.163 --> 14:42.030 maybe I raise my eye in a particular 14:42.039 --> 14:44.650 way and you can build up a model of , 14:44.659 --> 14:47.010 of speaking of a of a particular person 14:47.020 --> 14:49.131 speaking that when you then there's a 14:49.131 --> 14:51.242 purported video of like when Zelensky 14:51.242 --> 14:53.020 said that you Ukrainians should 14:53.020 --> 14:55.242 surrender purportedly , you can run the 14:55.242 --> 14:57.464 model that you've built up against that 14:57.464 --> 14:59.687 purported video and show like no , this 14:59.687 --> 15:01.687 is not at all the way that Zelensky 15:01.687 --> 15:01.419 would normally say that . And so that a 15:01.429 --> 15:03.651 layperson could see for themselves that 15:03.651 --> 15:05.929 the micro expressions are not accurate . 15:05.929 --> 15:07.651 So , you know , DARPA has been 15:07.651 --> 15:09.540 detecting developing that kind of 15:09.540 --> 15:11.818 technology um in the SEMA four program . 15:11.818 --> 15:14.820 So that's one part of the one part of 15:14.830 --> 15:17.052 the tool kit . Another part of the tool 15:17.052 --> 15:19.163 kit is having working with vendors to 15:19.163 --> 15:21.386 uh include watermarks and other kind of 15:21.386 --> 15:23.497 technology in there . Um When they do 15:23.497 --> 15:25.497 the generation or um we worked with 15:25.497 --> 15:27.608 NVIDIA when they produced the style G 15:27.608 --> 15:29.663 uh generator of , of still images so 15:29.663 --> 15:31.886 that there was a detector um that DARPA 15:31.974 --> 15:35.224 performers had like 99.999% 15:35.234 --> 15:37.765 accuracy . And that detector NVIDIA 15:37.775 --> 15:40.053 delayed the release of the style again , 15:40.053 --> 15:41.997 three generator so that they could 15:41.997 --> 15:44.219 release the detector at the same time . 15:44.219 --> 15:45.886 So this is like a practice of 15:45.886 --> 15:47.997 responsible release , you release the 15:47.997 --> 15:50.219 detector at the same time , you release 15:50.219 --> 15:52.219 the generator and then , you know , 15:52.219 --> 15:54.164 watermarking technology people are 15:54.164 --> 15:53.900 working on that . There's a recent 15:53.909 --> 15:56.210 result that suggests that watermarks 15:56.219 --> 15:58.380 can always be removed . Um But maybe 15:58.390 --> 16:00.612 you , you watermark and you sign at the 16:00.612 --> 16:02.723 same time and maybe if you remove the 16:02.723 --> 16:04.612 watermark , there's evidence that 16:04.612 --> 16:06.779 you've removed the watermark . So this 16:06.779 --> 16:06.260 is an area where the technology is 16:06.270 --> 16:08.219 moving super fast and we're likely 16:08.229 --> 16:09.951 going to need a combination of 16:09.951 --> 16:12.062 detection , partnering with producing 16:12.062 --> 16:14.173 the generator and the detector at the 16:14.173 --> 16:16.507 same time , water marking , signing and , 16:16.507 --> 16:18.729 and , and policy kinds of things all at 16:18.729 --> 16:20.618 the same time along with consumer 16:20.618 --> 16:22.840 groups . It's , it's not an easy area . 16:22.840 --> 16:21.940 We're going to have to learn as a 16:21.950 --> 16:24.172 society as we go along , how we need to 16:24.172 --> 16:26.339 do things . Um in terms of the voice , 16:26.339 --> 16:28.506 you know , you should just think ahead 16:28.506 --> 16:30.672 and ask the person like , did we go on 16:30.672 --> 16:32.894 that vacation ? Uh you know , something 16:32.894 --> 16:35.117 that you can uh have a shared knowledge 16:35.117 --> 16:37.228 of that an attacker wouldn't possibly 16:37.228 --> 16:39.394 know . And so , and it's hard to think 16:39.394 --> 16:41.561 about that when you're scared . But if 16:41.561 --> 16:43.783 you think in advance , um and have that 16:43.783 --> 16:45.894 as a prepared response , then you can 16:45.894 --> 16:48.006 very quickly diffuse that situation . 16:48.006 --> 16:47.859 So , you know , anticipating and 16:47.869 --> 16:49.647 planning ahead is a really good 16:49.647 --> 16:51.925 strategy and that's what we need to do . 16:51.925 --> 16:51.849 I think with manipulated media as a 16:51.859 --> 16:54.137 follow on to this question . Obviously , 16:54.137 --> 16:56.248 water marking is an important topic . 16:56.248 --> 16:58.081 It is one of the elements in the 16:58.090 --> 17:01.239 recently issued executive order on A I 17:01.250 --> 17:03.472 from the White House . And where do you 17:03.472 --> 17:06.079 see the maturity of this technology ? 17:06.089 --> 17:08.229 Because apparently there is synthetic 17:08.239 --> 17:10.017 data , real data data that gets 17:10.020 --> 17:12.569 transferred from multiple places . So 17:12.579 --> 17:14.746 is the same watermark going to be like 17:14.746 --> 17:17.229 a PKI like a certificate . And how does 17:17.239 --> 17:19.350 that work when somebody runs transfer 17:19.350 --> 17:21.572 learning models create synthetic data ? 17:21.572 --> 17:23.517 Should that we mark watermark in a 17:23.517 --> 17:25.572 different manner , right ? There all 17:25.572 --> 17:25.469 kinds of things that we are still 17:25.479 --> 17:27.989 figuring out and evolve learning and 17:28.000 --> 17:30.111 should there be standards around it ? 17:30.111 --> 17:32.222 Should this issue a guidelines around 17:32.222 --> 17:34.444 water marking data when it gets used in 17:34.449 --> 17:36.671 an A I model ? And it's open ? I mean , 17:36.671 --> 17:38.893 anyone can chime in here . So I I think 17:38.893 --> 17:41.005 we don't know the answer to all these 17:41.005 --> 17:42.893 questions because I think the the 17:42.893 --> 17:45.171 technical what , what , what can we do ? 17:45.171 --> 17:45.020 What can be not defend , you know , uh 17:45.030 --> 17:47.890 what can be defended against ? These 17:47.900 --> 17:49.956 are tons of open technical questions 17:49.956 --> 17:52.067 that researchers are actively working 17:52.067 --> 17:54.233 on the paper about removing watermarks 17:54.233 --> 17:56.456 under certain assumptions um can always 17:56.456 --> 17:58.678 be done was just published three months 17:58.678 --> 18:00.844 ago . So I think it's a place where we 18:00.844 --> 18:02.900 need to do active research and , and 18:02.900 --> 18:05.067 what are the procedures and mechanisms 18:05.067 --> 18:05.060 and how can they be defeated ? How 18:05.069 --> 18:07.180 could they not be defeated is , is an 18:07.180 --> 18:09.236 active area of research ? So I think 18:09.236 --> 18:11.291 this is a place where we're going to 18:11.291 --> 18:13.458 need to have the , the researchers and 18:13.458 --> 18:13.400 the companies that are generating the 18:13.410 --> 18:15.569 models . Um And you know , policy , 18:15.579 --> 18:17.750 people have ongoing conversations to 18:17.760 --> 18:19.871 figure out what is the best way to be 18:19.871 --> 18:21.927 defending against these , these very 18:21.927 --> 18:24.038 real threats . And I think we need to 18:24.038 --> 18:26.093 have conversation with um you know , 18:26.093 --> 18:27.927 people and educate people in the 18:27.927 --> 18:30.093 communities about what is possible and 18:30.093 --> 18:32.204 what is not possible and to have more 18:32.204 --> 18:32.199 sophisticated consumers as well , you 18:32.209 --> 18:34.209 know , Bill , what would you guys ? 18:34.439 --> 18:36.550 Yeah , I , I like your answer there . 18:36.550 --> 18:38.661 One of the things I picked up in what 18:38.661 --> 18:40.883 you said was this notion of interaction 18:40.883 --> 18:43.050 to sort of rule out a fake . And as we 18:43.050 --> 18:45.217 were talking before that , that may be 18:45.217 --> 18:47.550 sort of uh what we use next to interact . 18:47.550 --> 18:49.328 You know , these technologies , 18:49.328 --> 18:51.495 especially generative A I represents a 18:51.495 --> 18:53.439 new , a new breed of technology in 18:53.439 --> 18:55.670 which interaction is sort of the main , 18:55.989 --> 18:58.045 main benefit . I mean , obviously it 18:58.045 --> 19:00.156 gives you answers that you want , but 19:00.156 --> 19:02.267 this , this , it's almost a tactile , 19:02.267 --> 19:04.378 the , the ability to interact with it 19:04.378 --> 19:06.545 and , and feel like you're interacting 19:06.545 --> 19:06.030 with another . I won't say it , 19:06.040 --> 19:07.984 intelligence but another , another 19:07.984 --> 19:10.170 thing is palpable and it changes the 19:10.180 --> 19:12.449 way that uh you engage with the 19:12.459 --> 19:15.180 technology . But that interaction is 19:15.189 --> 19:17.719 perhaps what we can use to ferret out 19:17.729 --> 19:20.300 fake capabilities and also understand 19:20.310 --> 19:22.532 where they need to go , you know , II I 19:22.532 --> 19:24.866 interact with the chat model , you know , 19:24.866 --> 19:27.088 frequently . And uh at the beginning of 19:27.088 --> 19:29.254 it , it's , it's , it's , it's uh that 19:29.254 --> 19:31.254 any interaction , you can always go 19:31.254 --> 19:33.366 down a path and realize , well , it's 19:33.366 --> 19:32.729 getting off a little bit . I've , I've 19:32.739 --> 19:34.795 realized now that , that I'm dealing 19:34.795 --> 19:36.739 with something with some limits or 19:36.739 --> 19:38.795 something like that . Uh So I I just 19:38.795 --> 19:38.560 wanted to say that interaction is 19:38.569 --> 19:40.736 probably um something we could explore 19:40.736 --> 19:42.847 more . Yeah . And also , or the , you 19:42.847 --> 19:44.513 know , the provenance and the 19:44.513 --> 19:46.750 traceability also become critical . And 19:46.760 --> 19:48.816 although there might be technologies 19:48.816 --> 19:51.038 such as Blockchain technologies , it is 19:51.038 --> 19:53.260 the data that you put in the block that 19:53.260 --> 19:55.482 actually matters if you can't trust the 19:55.482 --> 19:54.540 data in the block . Obviously , 19:54.550 --> 19:56.883 Blockchain technology is not gonna help . 19:56.883 --> 19:59.430 So being able to have the provenance is 19:59.439 --> 20:01.469 critical . Yeah , I'm I'm thinking 20:01.479 --> 20:05.369 about A I as three dimensions like xyz 20:05.380 --> 20:07.819 axis , this data , this platforms and 20:07.829 --> 20:09.885 there's algorithms and some of these 20:09.885 --> 20:12.051 capabilities may need to be built into 20:12.051 --> 20:13.940 the platform itself . So there is 20:13.940 --> 20:15.996 consistency in how things get done , 20:15.996 --> 20:18.162 right ? When we do governance , should 20:18.162 --> 20:20.218 we audit and model and when we audit 20:20.218 --> 20:22.329 and model , does it say these are all 20:22.329 --> 20:22.099 the data sets you've used in your A I 20:22.109 --> 20:24.369 model and these are the watermark data 20:24.380 --> 20:26.436 versus non watermark data . I'm sure 20:26.790 --> 20:28.734 will be very busy for the next few 20:28.734 --> 20:30.957 years figuring out all these things and 20:30.957 --> 20:33.068 that works for current data . But you 20:33.068 --> 20:35.068 have it like you , you find a whole 20:35.068 --> 20:37.290 bunch of pictures somewhere . You still 20:37.290 --> 20:39.512 want to be able to use those pictures . 20:39.512 --> 20:38.890 But you know , were they generated and 20:38.900 --> 20:40.869 aged or those , you know , so it's 20:40.880 --> 20:42.824 tough , right ? The ability to say 20:42.824 --> 20:44.824 difference between fake and real is 20:44.824 --> 20:44.719 almost tough . I did some work with 20:44.729 --> 20:47.250 NASA Ames for a Mars mission and it's 20:47.260 --> 20:50.160 almost impossible to see the difference 20:50.170 --> 20:52.281 between outlook of a generator versus 20:52.281 --> 20:54.503 discriminator . It's almost very , very 20:54.503 --> 20:56.670 similar , it makes it very challenging 20:56.670 --> 20:56.660 to decide what is real versus what is 20:56.670 --> 20:58.781 fake . So Bill , while we are on this 20:58.781 --> 21:01.060 topic of LL MS and generative A I , 21:02.130 --> 21:04.352 it's obviously big , right ? Since GP T 21:04.352 --> 21:06.297 three to GP T four , it's not like 21:06.297 --> 21:08.352 Windows , seven to Windows 10 , it's 21:08.352 --> 21:09.963 100 X increase in underlying 21:09.963 --> 21:11.852 infrastructure requirement . That 21:11.852 --> 21:14.019 itself is a major challenge . Most cio 21:14.019 --> 21:16.019 s are not prepared for that kind of 21:16.019 --> 21:18.019 scale and requirements . Unless you 21:18.019 --> 21:17.880 have a supercomputer or large HPC 21:17.890 --> 21:20.001 cluster , it becomes very challenging 21:20.001 --> 21:21.557 and we have to leverage the 21:21.560 --> 21:23.727 capabilities of our vendor partners to 21:23.727 --> 21:25.893 achieve our mission outcomes . So from 21:25.893 --> 21:28.329 that perspective , uh what what do you 21:28.339 --> 21:31.310 see the potential to transform many 21:31.319 --> 21:33.208 aspects of society and government 21:33.208 --> 21:36.310 operations ? How is CD A O helping dod 21:36.459 --> 21:38.939 leverage generative A I to address our 21:38.949 --> 21:40.949 mission needs . Yeah . No , I I 21:40.959 --> 21:43.015 mentioned task force Lima previously 21:43.015 --> 21:45.181 and it was , it's been stood up . It's 21:45.181 --> 21:47.292 within the dod CD A O is leading it , 21:47.292 --> 21:49.126 but it's a , it's a coalition of 21:49.126 --> 21:51.348 component services and agencies helping 21:51.348 --> 21:53.403 define this . It's , you know , it's 21:53.403 --> 21:56.119 number one goal if you will is to uh to 21:56.130 --> 21:58.040 help understand how it and how 21:58.050 --> 22:00.050 generative A I , which is clearly a 22:00.050 --> 22:02.050 power of powerful capability can be 22:02.050 --> 22:04.219 applied to dod missions . Um But we 22:04.229 --> 22:06.451 want to do it in a responsible way . We 22:06.451 --> 22:08.562 want to do it in a co ordinated way . 22:08.562 --> 22:10.451 Uh We don't want , you know , one 22:10.451 --> 22:12.396 portion of the dod to get ahead of 22:12.396 --> 22:14.451 another portion and then suffer from 22:14.451 --> 22:16.562 potential consequences from that . Um 22:16.562 --> 22:18.562 So task force , Lima is , is up and 22:18.562 --> 22:20.673 running . It's got a , a defined time 22:20.673 --> 22:22.618 limit of 18 months to , to put out 22:22.618 --> 22:24.729 guidance . Uh And in the process will 22:24.729 --> 22:27.540 be , is assessing uh nearly 200 use 22:27.550 --> 22:29.661 cases that have been collected across 22:29.661 --> 22:31.828 the department . Uh for how generative 22:31.828 --> 22:33.994 A I specifically large language models 22:33.994 --> 22:36.106 at this point can be leveraged in dod 22:36.106 --> 22:38.217 workflows . Uh We like to think about 22:38.217 --> 22:40.328 it across a couple of dimensions , at 22:40.328 --> 22:42.550 least . Uh one is scale , you want this 22:42.550 --> 22:44.550 capability to help you with scale , 22:44.550 --> 22:46.772 whether it's scale of data , scale of , 22:46.772 --> 22:48.939 of time , too fast , for instance . Um 22:48.939 --> 22:50.717 And then the other dimension is 22:50.717 --> 22:52.883 consequence and that helps you sort of 22:52.883 --> 22:55.050 sort out where would be an appropriate 22:55.050 --> 22:57.383 application of the large language model . 22:57.383 --> 22:59.550 For instance , uh when I first sort of 22:59.550 --> 23:01.494 thought of that , I , I thought of 23:01.494 --> 23:01.449 consequence in terms of , you know , 23:01.459 --> 23:03.900 the actual workflow , like it was it a , 23:04.260 --> 23:06.371 you know , end of a kill chain or was 23:06.371 --> 23:08.371 it some back office function ? I've 23:08.371 --> 23:10.593 since revised what I use consequence to 23:10.593 --> 23:12.371 describe and that is , it's the 23:12.371 --> 23:14.640 consequence of an LLM uh mistake or 23:14.650 --> 23:17.410 incorrect answer on your workflow . And 23:17.420 --> 23:19.531 how , how , how impactful that is for 23:19.531 --> 23:21.642 you to have to correct it by shifting 23:21.642 --> 23:23.976 that definition . If you will , it , it , 23:23.976 --> 23:26.087 it makes us focus on the fact that LL 23:26.087 --> 23:28.270 MS are part of a system , an overall 23:28.280 --> 23:30.550 system that gets leveraged . Uh It has 23:30.560 --> 23:32.671 to be part of a workflow . You've got 23:32.671 --> 23:34.838 to have guardrails around it . It is a 23:34.838 --> 23:36.893 powerful generative capability , but 23:36.893 --> 23:39.116 you never are um you know , absolved of 23:39.116 --> 23:41.338 the responsibility for acting upon what 23:41.338 --> 23:43.504 it , what it gives you . Um And so you 23:43.504 --> 23:45.671 mentioned maturity before . And so one 23:45.671 --> 23:47.782 of the things as part of task force , 23:47.782 --> 23:49.671 Lima that we're doing and this is 23:49.671 --> 23:51.838 really to help us engage with industry 23:51.838 --> 23:53.949 uh is to define a maturity model . We 23:53.949 --> 23:55.949 have a maturity model if you will , 23:55.949 --> 23:58.060 levels of autonomy that really helped 23:58.060 --> 23:59.838 the whole field move forward by 23:59.838 --> 24:02.650 specifying levels of autonomy for a car . 24:02.660 --> 24:04.827 And in this way , we can have a common 24:04.827 --> 24:07.930 dialogue for uh what we need . Uh So 24:07.939 --> 24:10.939 we've got , we had an RF I out for um 24:11.410 --> 24:13.739 for several , I think , six or so weeks 24:14.010 --> 24:16.239 on this topic and received a lot of 24:16.250 --> 24:18.194 submissions and we'll be holding a 24:18.194 --> 24:19.972 working session at our upcoming 24:19.972 --> 24:22.420 symposium in February to discuss this 24:22.430 --> 24:24.652 maturity model . What , what nature the 24:24.652 --> 24:26.486 model itself will take ? Is it a 24:26.486 --> 24:28.597 process ? Is it a series of metrics ? 24:28.597 --> 24:30.597 And the idea , as I said will be to 24:30.597 --> 24:32.486 help us have a common lexicon , a 24:32.486 --> 24:34.910 common metric or measuring stick with 24:34.920 --> 24:37.031 which we can articulate our needs for 24:37.031 --> 24:39.253 dod workflows so that industry can move 24:39.253 --> 24:41.364 forward and we can all agree on where 24:41.364 --> 24:43.476 they really are in capability there . 24:43.476 --> 24:45.531 That's a good one . In fact , we are 24:45.531 --> 24:45.229 looking at similar maturity model with 24:45.239 --> 24:47.589 an IC . So it will be good to ensure 24:47.599 --> 24:50.050 that both Dod and IC have a common way 24:50.060 --> 24:52.004 of measuring something in terms of 24:52.004 --> 24:54.171 maturity . And yeah , Dia team is part 24:54.171 --> 24:56.227 of task force , Lima . Thank you for 24:56.227 --> 24:58.393 your leadership and I'm hoping at some 24:58.393 --> 25:00.449 point we'll filter narrow down those 25:00.449 --> 25:02.616 use case and it eventually go up to DS 25:02.616 --> 25:04.727 D for acting on those things . That's 25:04.727 --> 25:06.782 right . Thank you , Bill Dr Shanay . 25:07.709 --> 25:09.820 Given the supply chain challenges for 25:09.820 --> 25:12.380 microelectronics , what is dod doing to 25:12.390 --> 25:14.920 assure access to emerging technologies 25:14.930 --> 25:17.119 for future war fighting capabilities ? 25:17.390 --> 25:18.779 Yeah . So you know , the 25:18.779 --> 25:20.612 microelectronics comments that I 25:20.612 --> 25:23.280 alluded to the new initiatives using 25:23.290 --> 25:25.512 chips and Science Act funding . It's $2 25:25.512 --> 25:27.719 billion . So they fired up . So 400 25:27.729 --> 25:30.510 million in this initial FI F I 23 was 25:30.520 --> 25:32.880 used to support the infrastructure . 25:33.250 --> 25:35.250 And then the goal here is to ensure 25:35.250 --> 25:37.528 that the emerging technologies such as , 25:37.528 --> 25:39.910 you know , edge intelligence , uh A I 25:39.920 --> 25:42.339 hardware , five G six G technologies , 25:42.349 --> 25:44.460 quantum technologies and so on and so 25:44.460 --> 25:46.689 forth are able to be supported right 25:46.699 --> 25:48.755 here in the US . The challenge we've 25:48.755 --> 25:50.729 had is all these research uh 25:50.739 --> 25:52.739 demonstrations are available in the 25:52.739 --> 25:54.906 labs , university labs , for example , 25:54.906 --> 25:57.017 start up small companies , you've got 25:57.017 --> 25:58.795 to rely on offshore prototyping 25:58.795 --> 26:01.130 facilities to do the lab prototyping 26:01.140 --> 26:03.420 and then the lab to five prototyping A 26:03.430 --> 26:06.170 and that's uh an impediment because uh 26:06.180 --> 26:08.069 you know , if you have to rely on 26:08.069 --> 26:10.589 offshore prototyping capabilities , it 26:10.599 --> 26:12.710 it it could take time . Uh And then , 26:12.710 --> 26:14.655 so we want to try and address that 26:14.655 --> 26:16.430 challenge by ons showing the 26:16.439 --> 26:18.717 prototyping that's critical , you know , 26:18.717 --> 26:22.020 onshore fabs can help the laboratories 26:22.030 --> 26:24.640 do even better with the fa prototyping 26:24.890 --> 26:26.946 across those various technical areas 26:26.946 --> 26:29.168 that I talked about . For example , you 26:29.168 --> 26:31.279 know , edge intelligence can we do in 26:31.279 --> 26:33.446 addition to inference , can we also do 26:33.446 --> 26:35.334 training at the edge ? You know , 26:35.334 --> 26:37.446 that's going to be transformative and 26:37.446 --> 26:37.260 we can do that with things like beyond 26:37.369 --> 26:39.480 Von Neumann computing , for example , 26:39.480 --> 26:41.591 you know , neuromorphic computing and 26:41.591 --> 26:43.313 so on and so forth . Those are 26:43.313 --> 26:45.536 tremendous opportunities that we have . 26:45.536 --> 26:45.479 But that initial lab to five 26:45.579 --> 26:47.690 prototyping is just the first step in 26:47.690 --> 26:49.579 that cycle if you were uh because 26:49.579 --> 26:51.690 you've got to scale up the technology 26:51.690 --> 26:53.968 so that requires even more investments . 26:53.968 --> 26:55.857 And when you think about hardware 26:55.857 --> 26:57.746 technologies , you know , there , 26:57.746 --> 26:59.968 there's not too many sources of funding 26:59.968 --> 27:01.635 for that prototyping facility 27:01.635 --> 27:03.801 capability and venture capital sources 27:03.801 --> 27:05.968 are obviously not interested to the la 27:05.968 --> 27:07.912 to an extent because the return on 27:07.912 --> 27:09.912 investment and hardware relative to 27:09.912 --> 27:12.023 software is obviously limited . So we 27:12.023 --> 27:14.023 want to try and provide the patient 27:14.023 --> 27:16.135 capital 400 million per year over the 27:16.135 --> 27:18.246 next five years to ensure that we can 27:18.246 --> 27:20.079 on show the prototyping , we can 27:20.079 --> 27:21.746 closely couple the laboratory 27:21.746 --> 27:23.579 prototyping with the fabrication 27:23.579 --> 27:25.750 prototyping . We can protect the IP 27:26.099 --> 27:28.449 here in the US and also help bolster 27:28.459 --> 27:30.681 the workforce , you know , the regional 27:30.681 --> 27:32.848 economies because the workforce is one 27:32.848 --> 27:34.792 of the biggest challenges we might 27:34.792 --> 27:36.959 onshore a lot of the capabilities here 27:36.959 --> 27:39.181 in the US . But if we don't support the 27:39.181 --> 27:41.403 workforce , we have failed . So even as 27:41.403 --> 27:43.459 we bring all the ons , you know , do 27:43.459 --> 27:45.515 all the ons showing here in terms of 27:45.515 --> 27:47.681 the prototyping capabilities , we also 27:47.681 --> 27:49.930 want to support the workforce to ensure 27:49.939 --> 27:52.161 that the ecosystem that we develop here 27:52.161 --> 27:55.560 is sustainable in the longer term . How 27:55.910 --> 27:57.966 I'm just going off the script here . 27:57.966 --> 28:00.077 But there's a lot of things happening 28:00.077 --> 28:02.188 here . This Chips Act Microelectronic 28:02.188 --> 28:04.299 Commons Office of Strategic Capital . 28:04.299 --> 28:04.130 There's a lot of money for public 28:04.140 --> 28:06.307 private partnership . How can industry 28:06.307 --> 28:08.140 take advantage of it ? All these 28:08.140 --> 28:10.251 opportunities that dod is offering to 28:10.251 --> 28:12.307 our industry partners and academia . 28:12.307 --> 28:14.140 Yeah . As an example , the Micro 28:14.140 --> 28:16.670 Electronics Commons uh all its funding 28:16.680 --> 28:19.880 to the OS D , it is executed through 28:19.890 --> 28:23.199 NSWC crane , naval Surface Water Center 28:23.280 --> 28:26.300 crane Indiana . And so they have set up 28:26.310 --> 28:28.819 an OT A or the Transactions authority 28:29.000 --> 28:31.300 with NST XL . That's the National 28:31.310 --> 28:33.477 Security Technology Accelerator . It's 28:33.477 --> 28:35.643 a nonprofit consortium . So that's the 28:35.643 --> 28:37.754 consortium that is actually executing 28:37.754 --> 28:39.866 the Micro Electronics Commons . So if 28:39.866 --> 28:41.754 you go to the NST Excel website , 28:41.754 --> 28:43.754 there's a ton of information that's 28:43.754 --> 28:45.421 available there on this Micro 28:45.421 --> 28:47.421 Electronics Commons opportunity . I 28:47.421 --> 28:49.643 referred to the nearly 240 million that 28:49.643 --> 28:51.810 has been invested very recently in the 28:51.810 --> 28:54.579 commons infrastructure going forward . 28:54.599 --> 28:56.710 They're going to invest another close 28:56.710 --> 28:59.560 to 280 million to support projects uh 28:59.569 --> 29:01.791 because the infrastructure obviously is 29:01.791 --> 29:03.791 gonna be doing even better with the 29:03.791 --> 29:05.958 projects that go with it . And then so 29:05.958 --> 29:08.865 the commons is often running . I would 29:08.875 --> 29:11.306 invite anyone here that is interested 29:11.316 --> 29:13.906 to visit the NST Excel website again , 29:13.916 --> 29:16.645 as I said , the call for infrastructure 29:16.656 --> 29:19.026 was back in November now . It's more , 29:19.036 --> 29:21.036 it's about to be announced with the 29:21.036 --> 29:23.203 call for projects . I mean , so that's 29:23.203 --> 29:25.314 the next opportunity that's available 29:25.314 --> 29:25.026 to pretty much any stakeholders that's 29:25.036 --> 29:27.312 interested in joining the hubs as we 29:27.322 --> 29:29.401 have called them . So the um micro 29:29.552 --> 29:31.663 Chronic Commons is actually a network 29:31.663 --> 29:34.832 of hubs and codes hubs are these 29:34.842 --> 29:37.009 regional organizations that are formed 29:37.009 --> 29:38.675 to demonstrate the lab to fab 29:38.675 --> 29:40.898 prototyping and codes are the foundries 29:40.898 --> 29:42.953 of the fabs where the prototyping is 29:42.953 --> 29:45.175 getting done and any stakeholders , any 29:45.175 --> 29:47.398 organizations is welcome to join any of 29:47.398 --> 29:49.564 the eight hubs that have been selected 29:49.564 --> 29:51.620 through the Michael Jonis comments . 29:51.620 --> 29:53.898 Thank you for the clarification . Yeah , 29:53.898 --> 29:53.498 please go ahead . You mentioned 29:53.508 --> 29:55.675 supporting the workforce . What has to 29:55.675 --> 29:57.952 happen to support the workforce ? Yeah . 29:57.952 --> 30:00.008 So what we have asked uh the commons 30:00.008 --> 30:02.317 hubs to do is tell us what is that 30:02.328 --> 30:04.338 model that will help develop the 30:04.348 --> 30:06.237 workforce ? You know , how do you 30:06.237 --> 30:08.459 incentivize students , for example , to 30:08.459 --> 30:10.681 join , you know , the workforce through 30:10.681 --> 30:12.737 microphonic obviously . So it stinks 30:12.737 --> 30:14.959 like , you know , introducing curricula 30:14.959 --> 30:16.959 within the universities , it stinks 30:16.959 --> 30:19.015 like , you know , there's actually a 30:19.015 --> 30:21.237 program called scale within our R and E 30:21.237 --> 30:23.292 Microts portfolio which supports the 30:23.292 --> 30:25.126 workforce development . It helps 30:25.126 --> 30:27.348 undergraduate students get trained in , 30:27.348 --> 30:30.293 for example , radi micros and then they 30:30.303 --> 30:32.423 actually have opportunities to be 30:32.433 --> 30:35.203 interned in either a defense lab or 30:35.303 --> 30:37.525 maybe a government lab . And so through 30:37.525 --> 30:39.581 those kinds of opportunities , we've 30:39.581 --> 30:41.636 actually ensured that these hubs and 30:41.636 --> 30:43.803 courts will build the workforce in the 30:43.803 --> 30:46.300 future . Thank you . Thank you , Doctor 30:46.410 --> 30:49.270 Shenoy , Doctor Fisher . What are some 30:49.280 --> 30:53.250 of the maybe I'll let's 30:53.260 --> 30:55.316 stick with the A I thing . I want to 30:55.316 --> 30:57.482 get a little bit more deeper , right ? 30:57.482 --> 30:57.469 The other one is an A I question . Yeah , 30:57.479 --> 30:59.479 I know , I I wanna make sure , no , 30:59.479 --> 31:01.535 that's not the thing . I , I want to 31:01.535 --> 31:03.757 make sure to go back to this because of 31:03.757 --> 31:03.000 the , I want , I'm checking the time 31:03.010 --> 31:05.121 here . We want to give 15 minutes for 31:05.121 --> 31:08.430 our audience . So what 31:08.810 --> 31:10.829 area of A I is likely to make 31:10.839 --> 31:12.920 tremendous difference in cyber 31:12.930 --> 31:15.041 operations because that was the theme 31:15.041 --> 31:17.208 that we heard during this conference , 31:17.208 --> 31:19.780 both IO and I A and or in helping 31:19.790 --> 31:22.469 defend our digital systems . Can you do 31:22.479 --> 31:24.479 an update on what DARPA is doing in 31:24.479 --> 31:28.050 this space ? Yeah . So um I uh I 31:28.060 --> 31:30.599 think that A I will in fact have be 31:30.609 --> 31:33.630 quite transformative in the cyberspace . 31:33.890 --> 31:36.310 Uh One of the programs that DARPA is 31:36.319 --> 31:38.359 running right now is the A IC C 31:38.560 --> 31:40.393 challenge . So we announced this 31:40.393 --> 31:43.530 challenge at Black Hat in Las Vegas in 31:43.540 --> 31:46.359 August . Um And the semifinals will be 31:46.369 --> 31:49.209 at defcon in uh this summer , next , 31:49.219 --> 31:51.330 the coming summer and the finals will 31:51.330 --> 31:53.386 be the one after . And the idea is , 31:53.386 --> 31:57.060 can we use A I to help us find and fix 31:57.069 --> 31:58.958 vulnerabilities in infrastructure 31:58.958 --> 32:01.180 software so that the motivation more or 32:01.180 --> 32:03.459 less is Il Hein's testimony to Congress 32:03.469 --> 32:05.525 in February of last year that in the 32:05.525 --> 32:07.802 event that China were to invade Taiwan , 32:07.802 --> 32:09.913 it's very likely they would carry out 32:09.913 --> 32:11.969 destructive cyber attacks against us 32:11.969 --> 32:14.080 civilian infrastructure . Um and that 32:14.080 --> 32:16.025 they are very likely to be able to 32:16.025 --> 32:18.247 carry out , you know , that they can do 32:18.247 --> 32:20.080 that . I think that there was an 32:20.080 --> 32:19.739 article in the Washington Post 32:19.750 --> 32:21.920 yesterday or the day before about um 32:21.930 --> 32:24.270 pre placement in various places that 32:24.280 --> 32:26.280 suggest that it , you know , that , 32:26.280 --> 32:28.502 that it's providing supporting evidence 32:28.502 --> 32:30.558 to that . Um One of the things about 32:30.558 --> 32:32.780 our infrastructure software is a lot of 32:32.780 --> 32:34.947 it is run on open source software . Um 32:34.969 --> 32:37.280 And so the the competition is in fact 32:37.290 --> 32:39.599 to go and look at open source software 32:39.609 --> 32:41.530 and see if we can use A I enabled 32:41.540 --> 32:43.707 generative kinds of models to find and 32:43.707 --> 32:45.707 fix software . Uh One of the things 32:45.707 --> 32:47.929 about the generative models that I find 32:47.959 --> 32:50.704 shocking is that they can really 32:50.714 --> 32:52.658 understand software that transform 32:52.658 --> 32:54.770 transformer models , they're really , 32:54.770 --> 32:57.425 really good at language and code is 32:57.435 --> 33:00.905 just another kind of language . And um 33:00.915 --> 33:02.971 so I my background is in programming 33:02.971 --> 33:05.026 languages and , and writing code and 33:05.026 --> 33:07.193 analyzing code . And I'm blown away by 33:07.193 --> 33:09.248 the capacity of generative models to 33:09.248 --> 33:11.180 understand software . One of the 33:11.189 --> 33:13.133 motivations for this program was a 33:13.133 --> 33:15.189 paper that came out maybe six months 33:15.189 --> 33:17.133 ago at this point , maybe a little 33:17.133 --> 33:19.300 longer than the time goes by so fast . 33:19.300 --> 33:21.520 And the paper was comparing um I think 33:21.530 --> 33:25.349 it was GP T four at the task of finding 33:25.359 --> 33:27.470 and fixing vulnerabilities to bespoke 33:27.470 --> 33:29.470 to for this purpose . And GP T four 33:29.470 --> 33:31.415 came out roughly even with bespoke 33:31.415 --> 33:33.581 tools that were written by researchers 33:33.581 --> 33:35.803 who had spent years writing these tools 33:35.803 --> 33:37.915 to find and fix vulnerabilities . And 33:37.915 --> 33:39.803 it came out roughly even which is 33:39.803 --> 33:41.859 shocking , right ? And the thing was 33:41.859 --> 33:41.645 like the most common answer that GP T 33:41.655 --> 33:43.488 four gave was like , I need more 33:43.494 --> 33:45.438 information . And the thing that's 33:45.438 --> 33:48.530 amazing about foundation models is that 33:48.540 --> 33:50.630 you could just talk to them in human 33:50.640 --> 33:52.751 native form , you can talk to them in 33:52.751 --> 33:54.862 English . You could say like , well , 33:54.862 --> 33:57.029 what more information would you like ? 33:57.029 --> 33:59.084 And you could then answer it in like 33:59.084 --> 34:01.251 English . So DARPA had run a program a 34:01.251 --> 34:01.160 while ago about chess , which is about 34:01.170 --> 34:05.079 can we pair people with analysis 34:05.089 --> 34:07.089 tools about code to get kind of the 34:07.089 --> 34:09.089 best of human analysis expertise in 34:09.089 --> 34:11.367 looking at code with analysis programs . 34:11.367 --> 34:13.589 But these bespoke tools for finding and 34:13.589 --> 34:15.811 fixing to do a better job than just the 34:15.811 --> 34:17.756 tools or just the people . And the 34:17.756 --> 34:19.645 answer was kind of yes , somewhat 34:19.645 --> 34:21.867 successful . But that interface between 34:21.867 --> 34:23.756 the human expert and the code was 34:23.756 --> 34:25.978 really , really tough because the terms 34:25.978 --> 34:28.145 that people used and the analysis data 34:28.145 --> 34:30.702 structures that the code used were like 34:30.712 --> 34:33.193 really far apart . But chat EPT can 34:33.202 --> 34:35.472 talk English . Oh my God , it can talk 34:35.482 --> 34:37.649 English . This is amazing . So the the 34:37.649 --> 34:40.393 paper basically when it asked for more 34:40.403 --> 34:42.570 information , the people could talk to 34:42.570 --> 34:44.625 it in human native form and actually 34:44.625 --> 34:46.625 give it more information . And that 34:46.625 --> 34:48.959 allowed the tool to find lots more bugs . 34:48.959 --> 34:48.768 So that was the the intuition behind 34:48.778 --> 34:50.834 starting this competition . So we're 34:50.834 --> 34:52.945 really excited about this competition 34:52.945 --> 34:56.238 about partnering um preparing up the 34:56.258 --> 34:59.228 generative models with analysis tools 34:59.238 --> 35:01.294 to be able to find and fix bugs . At 35:01.294 --> 35:03.349 scale . And the question is , well , 35:03.349 --> 35:05.516 how do we stay at the forefront of the 35:05.516 --> 35:07.627 generative model kind of technology ? 35:07.627 --> 35:09.794 Because as you know , generative model 35:09.794 --> 35:11.960 capabilities are going so fast , super 35:11.960 --> 35:14.016 hard . Well , the answer there is we 35:14.016 --> 35:13.902 are partnering with open A I , 35:13.912 --> 35:15.745 Anthropic Google and Microsoft , 35:15.745 --> 35:17.856 they're going to give access to their 35:17.856 --> 35:19.968 models to people who are competing in 35:19.968 --> 35:19.882 this competition . So people who are 35:19.892 --> 35:22.003 competing in this competition will be 35:22.003 --> 35:23.948 able to get access to the frontier 35:23.948 --> 35:25.670 models at scale throughout the 35:25.670 --> 35:27.836 competition . So stay tuned . Um , the 35:28.001 --> 35:30.201 registration to be able to register to 35:30.211 --> 35:32.378 say that your team is competing open , 35:32.378 --> 35:34.711 I think yesterday . Um So we're really 35:34.721 --> 35:36.777 excited to see how that goes again . 35:36.777 --> 35:38.888 Finals semifinals will be a black hat 35:38.888 --> 35:41.054 in , in Las Vegas . This summer finals 35:41.054 --> 35:43.176 will be the following year . Um So 35:43.186 --> 35:45.242 hopefully this will allow us to find 35:45.242 --> 35:47.756 bugs at speed and scale and find and 35:47.766 --> 35:49.488 fix them . 01 more thing we're 35:49.488 --> 35:51.544 partnering with open source software 35:51.544 --> 35:53.710 foundation who is responsible for like 35:53.710 --> 35:55.877 when you find bugs in this open source 35:55.877 --> 35:55.696 software foundation , you submit them 35:55.706 --> 35:57.817 to them who then they're the ones who 35:57.817 --> 36:00.039 like go through and say yes , OK . This 36:00.039 --> 36:02.262 is actually a bug fix and they are then 36:02.262 --> 36:04.428 the ones that push out the bugs to get 36:04.428 --> 36:06.595 them fixed . Um , so they will be like 36:06.595 --> 36:08.817 watching this process to figure out how 36:08.817 --> 36:11.039 we can , um , get the bugs patched into 36:11.039 --> 36:13.095 the software at speed . And scale so 36:13.095 --> 36:12.449 that , you know , we don't envision in 36:12.459 --> 36:14.280 the long run or in the , in the 36:14.290 --> 36:16.457 competition , they will be , they will 36:16.457 --> 36:18.457 be patched automatically . We don't 36:18.457 --> 36:20.512 think that that would happen in real 36:20.512 --> 36:22.734 life unless there was a crisis . But we 36:22.734 --> 36:24.846 do imagine that we will try to figure 36:24.846 --> 36:26.901 out how to get that as smooth and as 36:26.901 --> 36:28.957 fast as possible so that we get bugs 36:28.957 --> 36:30.901 fixed , um , expeditiously even in 36:30.901 --> 36:33.068 short of a of a crisis . So probably a 36:33.068 --> 36:32.885 longer answer than you wanted . But 36:33.225 --> 36:35.447 that's exciting , right ? Because these 36:35.447 --> 36:37.614 kind of things are both good and bad . 36:37.614 --> 36:39.781 Now , a bad actor with access to these 36:39.781 --> 36:41.781 tools can generate TT PS and create 36:41.781 --> 36:43.892 exploit against those vulnerabilities 36:43.892 --> 36:46.840 potentially , right ? And so from that 36:46.850 --> 36:49.072 perspective , I want to go a little bit 36:49.072 --> 36:51.128 more deeper on they can already find 36:51.128 --> 36:53.294 software supply chain assurance . What 36:53.294 --> 36:55.517 do you think we should be doing or what 36:55.517 --> 36:57.406 is DARPA doing to ensure software 36:57.406 --> 36:59.379 supply chain assurance ? So , uh a 36:59.389 --> 37:01.556 number of different things . So one is 37:01.556 --> 37:03.810 we have uh an A IE on uh social cyber 37:03.820 --> 37:07.030 which is about analyzing the um 37:07.590 --> 37:09.949 the supply chain itself , the like the 37:10.189 --> 37:12.022 the github kind of tool chain to 37:12.022 --> 37:14.189 understand who is committing and , and 37:14.189 --> 37:16.411 where they're committing to . Uh and it 37:16.411 --> 37:18.467 has found lots of really interesting 37:18.467 --> 37:20.522 things . So , um so that's one thing 37:20.522 --> 37:20.219 like understanding this , the social 37:20.229 --> 37:22.550 processes behind the code development 37:22.686 --> 37:25.656 um to , to understand when are people 37:25.666 --> 37:27.888 committing that might be committing for 37:27.888 --> 37:29.833 nefarious purposes instead of good 37:29.833 --> 37:31.722 purposes . Um That was actually a 37:31.722 --> 37:33.722 pretty small effort that had really 37:33.722 --> 37:33.676 good return on investment . So , so 37:33.686 --> 37:35.996 there's that , um but then on the other 37:36.006 --> 37:38.228 side , the resilient um systems part of 37:38.228 --> 37:40.726 our tool chain , um we have been 37:40.736 --> 37:43.375 working on applying formal methods to 37:43.385 --> 37:45.607 can we build software that is much more 37:45.607 --> 37:47.829 resilient , that is built a much higher 37:47.829 --> 37:49.829 standard of correctness . The hacks 37:49.829 --> 37:51.829 program which was started in 2011 , 37:51.829 --> 37:54.107 started with the premise of , you know , 37:54.107 --> 37:56.329 formal methods have been promised since 37:56.329 --> 37:58.441 age and memorial for 50 years that we 37:58.441 --> 37:57.461 could build software that came with a 37:57.471 --> 38:00.011 proof of correctness and for that 50 38:00.021 --> 38:02.077 years they failed . Um but the hacks 38:02.077 --> 38:04.354 program demonstrated that no , in fact , 38:04.354 --> 38:06.465 they are ready for prime time that we 38:06.465 --> 38:08.410 can build systems that come with a 38:08.410 --> 38:10.354 proof of correctness that are much 38:10.354 --> 38:09.741 harder to hack into . So in that 38:09.751 --> 38:12.030 program , at the , at the beginning of 38:12.040 --> 38:14.262 the program , hackers were able to hack 38:14.262 --> 38:16.429 into a helicopter , no problem , which 38:16.429 --> 38:18.540 is not surprising . At the end of the 38:18.540 --> 38:20.879 program , they hackers were charged 38:20.889 --> 38:22.833 with breaking into and attacking a 38:22.833 --> 38:25.000 helicopter while it was in flight with 38:25.000 --> 38:27.167 test pilots on board . And the hackers 38:27.167 --> 38:29.222 were not able to hack in so good for 38:29.222 --> 38:31.389 the test pilots , right ? Showing that 38:31.389 --> 38:33.222 really like formal methods based 38:33.222 --> 38:35.333 technology come a super long way . We 38:35.333 --> 38:37.278 actually have tons of capabilities 38:37.278 --> 38:39.333 about how to build systems to a much 38:39.333 --> 38:41.556 higher standard than we do these days . 38:41.556 --> 38:43.667 And the challenge is actually finding 38:43.667 --> 38:45.833 the will to apply things like verified 38:45.833 --> 38:47.945 parcels , things like . So four micro 38:47.945 --> 38:49.889 kernel architectural strategies to 38:49.889 --> 38:52.222 build systems to a much higher standard . 38:52.222 --> 38:54.389 We're just not doing it . A I actually 38:54.389 --> 38:56.372 is helpful there to allow us to 38:56.382 --> 38:58.549 automate a lot of those formal methods 38:58.549 --> 39:00.771 based approaches so that we can do them 39:00.771 --> 39:02.882 faster . Uh I'll stop talking because 39:02.882 --> 39:02.431 I've been talking for way too long . 39:02.441 --> 39:04.663 But that like we actually do know a lot 39:04.663 --> 39:06.663 about how to do this and what we're 39:06.663 --> 39:08.774 currently lacking is the requirements 39:08.774 --> 39:08.741 to , to go do so . But you know , 39:08.751 --> 39:10.751 adversaries might motivate us to do 39:10.751 --> 39:12.807 that sooner rather than later . No , 39:12.807 --> 39:15.029 thank you for that . Well , thought out 39:15.029 --> 39:14.481 answer . Thank you very much , Bill . 39:14.491 --> 39:16.713 Let's go to you . I mean , we have some 39:16.713 --> 39:18.880 challenges here . Technology is moving 39:18.880 --> 39:21.047 at a very fast pace . We have concerns 39:21.047 --> 39:23.102 on supply chain , both from physical 39:23.102 --> 39:25.158 supply chain , from microelectronics 39:25.158 --> 39:27.269 and the software supply chain for the 39:27.269 --> 39:27.199 assurance of our systems and networks . 39:27.429 --> 39:30.649 So what's your definition of decision 39:30.659 --> 39:32.659 advantage ? I mean , we are here in 39:32.659 --> 39:34.992 national security , we want to win wars , 39:34.992 --> 39:37.270 we want to prevent strategic surprises . 39:37.270 --> 39:39.830 So where is CD EO from your stated goal 39:39.840 --> 39:42.215 for accelerating the option of data 39:42.225 --> 39:44.975 analytics and A I ? Right . So decision 39:44.985 --> 39:46.763 advantage is the term we use to 39:46.763 --> 39:49.375 describe sort of uh what the benefit is 39:49.385 --> 39:51.163 of leveraging , you know , data 39:51.163 --> 39:53.329 analytics and A I why we , why we want 39:53.329 --> 39:55.218 all this pain , if you will to go 39:55.218 --> 39:57.385 through to , to modernize because it's 39:57.385 --> 39:59.552 not magic Pixie dust , it's gonna take 39:59.552 --> 40:01.663 work and most of that work will be in 40:01.663 --> 40:03.774 figuring out where your data is and , 40:03.774 --> 40:05.885 and figuring out how to leverage it . 40:05.885 --> 40:07.996 But so decision in our strategy which 40:07.996 --> 40:10.218 was recently released and signed off by 40:10.218 --> 40:10.040 the deputy decision advantage 40:10.050 --> 40:12.100 translates into things like battle 40:12.110 --> 40:14.590 space awareness , uh uh more resilient 40:14.600 --> 40:17.360 and fast closing of kill chains , uh 40:17.370 --> 40:20.379 better logistic support , uh business 40:20.389 --> 40:22.560 management and things like that . Um 40:22.570 --> 40:24.870 You know , we were talking beforehand , 40:25.209 --> 40:27.098 one of the ways that people often 40:27.098 --> 40:29.489 describe the use of uh data analytics 40:29.500 --> 40:31.500 and A I certainly in the enterprise 40:31.500 --> 40:33.667 space is helping people , commanders , 40:33.667 --> 40:35.889 people in charge know where their stuff 40:35.889 --> 40:37.833 is because that's very important . 40:37.833 --> 40:40.000 Obviously , you know , um knowing that 40:40.000 --> 40:42.222 material is gonna show up when you need 40:42.222 --> 40:44.333 it , knowing where your troops are uh 40:44.333 --> 40:46.056 is very important and data and 40:46.056 --> 40:48.333 analytics can help you a lot with that . 40:48.333 --> 40:50.556 And A I can help you in predicting when 40:50.556 --> 40:52.667 your stuff might break for instance . 40:52.667 --> 40:54.500 But to get true to this decision 40:54.500 --> 40:56.444 advantage , you need to know where 40:56.444 --> 40:58.278 their stuff is too . Uh And that 40:58.278 --> 41:00.611 requires something other than analytics . 41:00.611 --> 41:03.199 It requires A I . Uh because A I is 41:03.209 --> 41:05.098 something that needs to interpret 41:05.098 --> 41:06.987 signals , whether they're imagery 41:06.987 --> 41:09.098 signals , whether they're other types 41:09.098 --> 41:11.209 of signals uh that aren't spelled out 41:11.209 --> 41:13.265 like things might be in a in tabular 41:13.265 --> 41:16.260 tabular presented data . So um 41:16.629 --> 41:19.560 this motivation for the use of A I uh 41:19.570 --> 41:21.479 enables you to realize decision 41:21.489 --> 41:23.545 advantage from that perspective . It 41:23.545 --> 41:25.656 it's almost like you need to see both 41:25.656 --> 41:27.822 uh you know , the white pieces and the 41:27.822 --> 41:29.822 black pieces in a on a chessboard , 41:29.822 --> 41:31.989 that kind of thing . You can't do just 41:31.989 --> 41:34.100 one side of it . So this but decision 41:34.100 --> 41:33.959 advantage is really where we think this , 41:33.969 --> 41:36.560 this helps because in the end , uh the 41:36.570 --> 41:39.040 human is involved in all decisions 41:39.050 --> 41:42.139 related to taking action and to do to 41:42.149 --> 41:44.260 make the best decisions . You need an 41:44.260 --> 41:46.482 advantage in data analytics and A I can 41:46.482 --> 41:48.705 provide that for you . So it looks like 41:48.705 --> 41:50.538 we might be able to leverage the 41:50.538 --> 41:52.649 multimodal LL MS and the large models 41:52.649 --> 41:54.840 coming in for effective human machine 41:54.850 --> 41:57.469 teaming , leveraging the next 41:57.479 --> 42:00.060 generation digital platforms , right ? 42:00.070 --> 42:02.181 That's right . I I mean , I think the 42:02.181 --> 42:04.929 previ way this conversational interface 42:04.939 --> 42:06.939 allows you to explore that in a way 42:06.939 --> 42:09.050 that you couldn't explore before . Um 42:09.050 --> 42:11.217 There are a lot of things I think that 42:11.217 --> 42:13.106 that we do which are difficult to 42:13.106 --> 42:15.272 explain that we explain after the fact 42:15.272 --> 42:17.328 that these models will help us get a 42:17.328 --> 42:19.439 window into . Uh I I mean , I some of 42:19.439 --> 42:21.495 it is being done right now but being 42:21.495 --> 42:23.550 able to do that at a large scale , I 42:23.550 --> 42:25.717 think uh represents a transformational 42:25.717 --> 42:28.550 capability . Thank you doctor . What 42:28.560 --> 42:30.250 are some examples of emerging 42:30.260 --> 42:32.429 technological capabilities that should 42:32.439 --> 42:35.060 guide DU D investments . And how is 42:35.070 --> 42:37.590 duty re establishing microelectronics 42:37.600 --> 42:40.350 technology leadership in the US that 42:40.360 --> 42:42.750 has largely shifted to East Asia . Yeah . 42:42.959 --> 42:45.070 Yeah . So , you know , when you think 42:45.070 --> 42:47.070 about micro electronics , uh it's a 42:47.070 --> 42:49.292 fairly broad endeavor , right ? Because 42:49.292 --> 42:51.015 it's not just about processing 42:51.015 --> 42:51.010 technology , it's about memory 42:51.020 --> 42:52.530 technologies , it's about 42:52.540 --> 42:54.484 communications technologies , it's 42:54.484 --> 42:56.770 about radar applications . And so 42:56.780 --> 42:59.002 there's a number of things that we need 42:59.002 --> 43:01.224 to address . And so we have through the 43:01.224 --> 43:03.391 commons identified six technical areas 43:03.510 --> 43:05.919 that are critical for future dod S 43:05.929 --> 43:08.050 applications and needs uh things like 43:08.060 --> 43:11.320 five G six G technologies A I hardware 43:11.330 --> 43:13.649 quantum technologies . Commercial leap 43:13.659 --> 43:15.715 ahead technologies , commercial leir 43:15.715 --> 43:17.937 technologies are this is a fairly broad 43:17.937 --> 43:19.770 category . Uh Things like gallum 43:19.770 --> 43:21.937 nitrate technology is one example of a 43:21.937 --> 43:23.770 commercial leap ahead technology 43:23.770 --> 43:25.826 that'll have transformative benefits 43:25.826 --> 43:25.580 for radar applications being able to 43:25.590 --> 43:28.199 see other uh being able to detect 43:28.209 --> 43:30.153 faster . So those are the kinds of 43:30.159 --> 43:32.215 technologies that we want to support 43:32.215 --> 43:33.770 when it comes to processing 43:33.770 --> 43:36.080 technologies , literally for 6 to 7 43:36.090 --> 43:38.219 decades . Ever since the inception of 43:38.229 --> 43:40.719 this industry , we've relied on what's 43:40.729 --> 43:43.399 known as the W Neumann architecture , 43:43.409 --> 43:45.631 which is where the logic and the memory 43:45.631 --> 43:47.409 are separated . Now , that's an 43:47.409 --> 43:49.631 inefficient architecture because you've 43:49.631 --> 43:51.798 got shutter data between the logic and 43:51.798 --> 43:53.909 the memory chip . Uh And that's about 43:53.909 --> 43:56.020 1000 times more energy intensive than 43:56.020 --> 43:55.709 actually switching the transition 43:55.719 --> 43:57.941 energies . So uh we have to think about 43:57.941 --> 43:59.775 new paradigms if you will in new 43:59.775 --> 44:01.879 architectures , brain inspired 44:01.889 --> 44:04.056 architectures , Neuromorphic computing 44:04.056 --> 44:06.111 architectures is one good example of 44:06.111 --> 44:08.111 that where the logic and memory are 44:08.111 --> 44:10.278 very closely interlinked intertwined . 44:10.278 --> 44:12.389 You know in memory architectures near 44:12.389 --> 44:14.222 memory architectures , these new 44:14.222 --> 44:16.540 architect textures will enable a lot 44:16.550 --> 44:19.550 more efficiency in computing , enable 44:19.560 --> 44:21.985 the swap that's needed . And I refer to 44:21.995 --> 44:24.344 for example , uh being able to do 44:24.354 --> 44:26.554 training as well as inference at the 44:26.564 --> 44:28.735 edge today . The way edge intelligence 44:28.745 --> 44:31.814 works is you gotta have the uh data 44:31.824 --> 44:33.991 trained on a cloud , for example . And 44:33.991 --> 44:36.213 then you transfer all that onto an edge 44:36.213 --> 44:38.784 device uh that only allows for 44:38.794 --> 44:40.405 inference , but for military 44:40.405 --> 44:42.183 applications , you might have a 44:42.183 --> 44:44.238 situation where you need to train at 44:44.238 --> 44:46.072 the edge as well . So can you do 44:46.072 --> 44:48.294 training and inference at the edge in a 44:48.489 --> 44:51.030 uh sort of swap uh uh platform ? You 44:51.040 --> 44:53.262 know , that that's a critical challenge 44:53.262 --> 44:55.318 that we're trying to address . We're 44:55.318 --> 44:57.429 actually working on it . I mean , one 44:57.429 --> 44:59.262 of the areas we are looking as a 44:59.262 --> 45:01.096 community is how do we integrate 45:01.096 --> 45:02.984 unmanned capabilities into manned 45:02.984 --> 45:04.873 missions ? What does learning and 45:04.873 --> 45:04.419 reasoning on the edge mean in a 45:04.469 --> 45:06.358 disconnected environment when the 45:06.358 --> 45:08.580 signals are jammed ? How do we continue 45:08.580 --> 45:10.525 our mission mission ? How does the 45:10.525 --> 45:12.525 resiliency and mission assurance be 45:12.525 --> 45:14.580 increased with technologies like A I 45:14.580 --> 45:16.469 with neuromorphic processes ? And 45:16.469 --> 45:16.330 that's something we are actually 45:16.340 --> 45:18.879 working with our labs and putting into 45:18.889 --> 45:21.000 our operating environment . So you're 45:21.000 --> 45:22.722 absolutely right . That's very 45:22.722 --> 45:24.945 essential , very important and as we do 45:24.945 --> 45:27.000 that , we also need the foundational 45:27.000 --> 45:29.056 digital platform which has two chain 45:29.056 --> 45:31.370 automation MS model exchange , things 45:31.379 --> 45:33.101 like that where we can take an 45:33.101 --> 45:35.435 application and deploy it on a platform , 45:35.435 --> 45:37.657 whether it's an M 29 or a space vehicle 45:37.657 --> 45:39.712 or a submarine , doesn't matter . We 45:39.712 --> 45:41.657 need to have the underlying common 45:41.657 --> 45:45.080 platform for us to scale and work with 45:45.090 --> 45:47.034 our partners and allies around the 45:47.034 --> 45:49.090 world for joint missions And some of 45:49.090 --> 45:51.034 the work we are doing in Mars will 45:51.034 --> 45:52.701 support that we take our also 45:52.701 --> 45:54.534 foundation military intelligence 45:54.534 --> 45:56.590 provide to platforms like JC two for 45:56.590 --> 45:58.701 enabling multi domain operations . So 45:58.701 --> 46:00.868 that's a good one . And this is a very 46:00.868 --> 46:02.979 fast changing world and we need to be 46:02.979 --> 46:04.979 open minded challenge . Our mindset 46:04.979 --> 46:07.090 continue to collaborate to accelerate 46:07.090 --> 46:09.090 the speed and scale of innovation . 46:09.090 --> 46:11.312 With that . I'll ask one final question 46:11.312 --> 46:13.479 to Dr Fisher and then we'll open up it 46:13.479 --> 46:13.290 up to the audience for any questions to 46:13.300 --> 46:16.129 the panelists . So Dr Fisher , what are 46:16.139 --> 46:18.919 some of the most promising applications 46:18.929 --> 46:22.080 of A I technology you can foresee from 46:22.090 --> 46:24.146 a DARPA perspective , right ? Well , 46:24.146 --> 46:26.257 I've already talked about one finding 46:26.257 --> 46:28.423 and fixing bugs at speed and scale , I 46:28.423 --> 46:31.729 think uh co code code 46:31.739 --> 46:35.389 pilots um is another um which obviously 46:35.399 --> 46:37.732 we're already seeing in commercial ways . 46:37.732 --> 46:37.729 Although I think the quality of the 46:37.739 --> 46:40.530 code um if we can get that so that it's 46:40.540 --> 46:42.651 writing like verified code . I talked 46:42.651 --> 46:44.818 about how code is just a language . So 46:44.818 --> 46:47.110 are specification languages . So are um 46:47.120 --> 46:49.176 our proof assistant . So if we could 46:49.176 --> 46:51.398 get that to be writing verified code at 46:51.398 --> 46:53.453 speed and scale so that we could get 46:53.453 --> 46:55.620 like high quality code being generated 46:55.620 --> 46:57.453 really fast , that would be game 46:57.453 --> 46:59.564 changing . I think another is sort of 46:59.564 --> 47:01.731 lab assistance , science assistance so 47:01.731 --> 47:03.842 that we could get science accelerated 47:03.842 --> 47:03.725 that could help with like global 47:03.735 --> 47:05.457 warming and all sorts of other 47:05.457 --> 47:07.791 scientific challenges that are going on . 47:07.791 --> 47:10.074 Um I think there's actually no end of 47:10.084 --> 47:12.306 really amazing challenges that could be 47:12.306 --> 47:14.528 coming out of of A I I think we're just 47:14.528 --> 47:16.640 just starting to rub the genie bottle 47:16.640 --> 47:18.485 because of the transformative 47:18.495 --> 47:20.328 capabilities that are coming out 47:20.328 --> 47:22.384 because of uh generative A I I think 47:22.384 --> 47:24.551 maybe I'll stop there because we don't 47:24.551 --> 47:24.334 have very much time for questions , but 47:24.344 --> 47:28.010 there's , there's huge game changing uh 47:28.199 --> 47:30.310 technology that's gonna be coming out 47:30.310 --> 47:32.255 because of A I in partnership with 47:32.255 --> 47:34.199 other kinds of technologies . No , 47:34.199 --> 47:36.199 thank you very much . This has been 47:36.199 --> 47:36.149 exciting , great perspective from our 47:36.159 --> 47:38.326 panel members here . With that . We'll 47:38.326 --> 47:40.270 open it up to the audience for any 47:40.270 --> 47:42.492 questions to any of the panel members . 47:42.492 --> 47:42.389 Maybe we need some light to see who's 47:42.399 --> 47:43.510 asking the question 47:50.070 --> 47:52.292 today . We really appreciate it . There 47:52.292 --> 47:54.348 are such great leader and such great 47:54.348 --> 47:56.348 leaders in our organization . Uh We 47:56.348 --> 47:58.459 really appreciate that as we're going 47:58.459 --> 48:01.419 forward . Um You've all talked about 48:01.429 --> 48:03.596 some of the things that excite you and 48:03.596 --> 48:05.762 some of the things that discourage you 48:05.762 --> 48:07.762 about what we're doing . But if you 48:07.762 --> 48:09.873 could concentrate on one thing that's 48:09.873 --> 48:12.096 in your portfolio now that you think is 48:12.096 --> 48:14.262 the future that you want to put your , 48:14.262 --> 48:16.485 the most of your interest in as thought 48:16.485 --> 48:18.651 leaders , as those leaders for these , 48:18.651 --> 48:20.818 these activities . Where would you put 48:20.818 --> 48:23.040 most of your effort ? I'll go , I'll go 48:23.040 --> 48:25.439 first partnerships . Uh I would say uh 48:25.449 --> 48:27.393 the word ecosystem is brought up a 48:27.393 --> 48:29.505 couple of times . You mentioned it in 48:29.505 --> 48:30.949 terms of an ecosystem for 48:30.949 --> 48:33.060 microelectronics . Uh I , I believe , 48:33.060 --> 48:35.116 and I wanna put most of my energy in 48:35.116 --> 48:37.282 the development of uh an A I ecosystem 48:37.282 --> 48:39.338 that enables the dod to benefit from 48:39.338 --> 48:41.282 the innovation that's happening in 48:41.282 --> 48:43.449 industry and laboratories everywhere . 48:43.449 --> 48:45.760 Um The dod needs to needs technologies 48:45.770 --> 48:48.500 to apply to the missions of national 48:48.510 --> 48:50.860 defense and national security and 48:50.899 --> 48:53.590 innovation is happening outside the dod 48:53.600 --> 48:56.860 uh especially with regard to A I and an 48:56.870 --> 49:00.000 ecosystem can enable that to , to make 49:00.010 --> 49:02.899 mission impact . Um The dod has a big 49:02.909 --> 49:04.576 role there because we need to 49:04.576 --> 49:06.798 understand how this technology can have 49:06.798 --> 49:08.798 an impact . And as I said , get our 49:08.798 --> 49:10.742 data , right ? And we can help set 49:10.742 --> 49:12.465 standards for what we need the 49:12.465 --> 49:14.939 technology to do , but the ecosystem 49:14.949 --> 49:17.100 which is not only technology but is 49:17.110 --> 49:20.100 also people as well as data 49:20.510 --> 49:22.843 needs to be there for us to leverage it . 49:23.540 --> 49:25.873 Yes . So what I would say is , you know , 49:25.873 --> 49:27.984 the one thing that we absolutely need 49:27.984 --> 49:29.873 to do is to ensure the asymmetric 49:29.873 --> 49:32.320 advantage for our war fighter against 49:32.330 --> 49:34.386 our adversaries because without that 49:34.386 --> 49:36.608 we're going to be losing against uh you 49:36.608 --> 49:38.830 know , our , these powerful adversaries 49:38.830 --> 49:41.659 that are being uh you know , developed 49:41.739 --> 49:44.179 uh a and so that would be the focus 49:44.189 --> 49:46.479 trying to ensure that our war fighters 49:46.489 --> 49:48.656 have the asymmetric advantage . And if 49:48.656 --> 49:50.822 that means a few different things , so 49:50.822 --> 49:52.711 be it . But we absolutely have to 49:52.711 --> 49:54.656 ensure that we have the asymmetric 49:54.656 --> 49:58.330 capabilities uh to uh enter the , you 49:58.340 --> 50:01.590 know , us s uh leadership as well as an 50:01.600 --> 50:03.267 asymmetric advantage . Yeah . 50:05.199 --> 50:07.580 Um you know , my mission is , is the 50:07.590 --> 50:10.219 surprise . So scanning the horizon for 50:10.229 --> 50:13.090 potential um threats and opportunities 50:13.100 --> 50:15.322 in the surprise space . So that kind of 50:15.322 --> 50:17.433 mitigates against like focusing in on 50:17.433 --> 50:21.020 one particular thing . Um uh The thing 50:21.030 --> 50:23.252 that I'm most worried about actually is 50:23.252 --> 50:26.219 the , the um exposure of um 50:28.330 --> 50:30.497 our , our digital attack surface , you 50:30.497 --> 50:32.750 know , how exposed we are and how 50:32.760 --> 50:34.871 effectively China might be putting in 50:34.871 --> 50:36.982 implants all over the place . So that 50:36.982 --> 50:38.979 on day one of a of a conflict with 50:38.989 --> 50:41.169 China , it might just be like all of 50:41.179 --> 50:43.235 our system , you know , like kind of 50:43.235 --> 50:45.457 like when you try to turn on the system 50:45.457 --> 50:47.457 and it just goes , no , I'm kind of 50:47.457 --> 50:49.401 worried that that's where might be 50:49.401 --> 50:51.346 where we are and so what can we be 50:51.346 --> 50:53.568 doing to counteract that and so make it 50:53.568 --> 50:55.800 so that we can have that not happen . 50:56.149 --> 50:58.260 Um And so A IC C is , is part of that 50:58.260 --> 51:00.449 activity um building tools and 51:00.459 --> 51:02.459 techniques to make our systems much 51:02.459 --> 51:05.219 less uh re uh resistant to such attack 51:05.229 --> 51:07.062 or much more resilient , less uh 51:07.062 --> 51:09.370 susceptible , convincing people to 51:09.379 --> 51:11.546 apply the tools and techniques that we 51:11.546 --> 51:13.379 already have . Um And take those 51:13.379 --> 51:16.209 threats seriously . Um So that , that , 51:16.219 --> 51:18.850 yeah , my answer today . So and Mac , I 51:18.860 --> 51:21.790 would , I would say it's culture . We 51:21.800 --> 51:23.911 need a culture of innovation , growth 51:23.911 --> 51:26.133 mindset , challenge our assumptions and 51:26.133 --> 51:28.300 take some risks . Not taking a risk is 51:28.300 --> 51:31.330 a much bigger risk . So I would highly 51:31.679 --> 51:33.568 say that we have to invest in our 51:33.568 --> 51:35.401 people , take care of our people 51:35.401 --> 51:37.512 because that's what will give us that 51:37.512 --> 51:39.568 strategic asymmetric advantage going 51:39.568 --> 51:39.219 forward because technologies keep 51:39.229 --> 51:41.451 changing and we have to be prepared and 51:41.451 --> 51:43.673 from a preparedness standpoint , it's a 51:43.673 --> 51:45.618 mindset and culture that's equally 51:45.618 --> 51:47.729 important as we continue to invest in 51:47.729 --> 51:50.879 technologies . OK . Any more 51:50.889 --> 51:53.510 questions ? I can't see who's asking 51:53.520 --> 51:54.520 the question . 51:58.080 --> 52:01.149 Is there anyone any questions doug do 52:01.159 --> 52:02.937 you have any questions ? We had 52:02.937 --> 52:05.103 questions about your interfaces ? OK . 52:05.103 --> 52:07.048 Ep has a question . Just got one . 52:12.090 --> 52:14.739 So from a dia perspective , ultimately 52:14.750 --> 52:17.110 be responsible for targeting , right ? 52:17.120 --> 52:19.530 Um One of the things that you said is 52:19.540 --> 52:22.979 when you go from A I 3.0 to 4.0 right ? 52:22.989 --> 52:24.899 The infrastructure is 100 times . 52:25.729 --> 52:28.110 Typically , we invest in new 52:28.120 --> 52:30.669 capabilities at the expense of 52:30.679 --> 52:34.229 foundational it , we are not typically 52:34.239 --> 52:36.406 data centric . We are more application 52:36.409 --> 52:39.340 centric . Uh Doctor Fisher , you said , 52:39.350 --> 52:42.429 hey , I'm already afraid on day one on , 52:42.439 --> 52:44.661 you know , where Chinese implants could 52:44.661 --> 52:48.590 take place . Um as you adopt more and 52:48.600 --> 52:51.500 more A I , you could also have counter 52:51.510 --> 52:54.010 A I . And you've already mentioned , I 52:54.020 --> 52:55.964 don't trust the difference between 52:55.964 --> 52:57.964 generative and differentiated A I . 52:57.964 --> 53:00.298 There's , you know , there are multiple , 53:00.298 --> 53:02.409 it's very hard to tell the difference 53:02.409 --> 53:04.576 between fake and if you're responsible 53:04.576 --> 53:08.010 for targeting right at the speed 53:08.020 --> 53:11.649 of mission . And when we talk about um 53:12.800 --> 53:16.520 A I Ethics , our adversary is not going 53:16.530 --> 53:19.729 to be using A I ethics to slow down 53:19.739 --> 53:21.961 what the end goal is . Where do you see 53:21.961 --> 53:23.961 this going ? 10 years from now ? 15 53:23.961 --> 53:26.183 years from now . So especially in the , 53:26.183 --> 53:28.350 in the against the backdrop of a China 53:28.350 --> 53:32.310 Taiwan conflict in either 27 39 53:32.320 --> 53:34.820 45 in 2025 2045 . 53:36.110 --> 53:38.166 Ep , that's a very common question . 53:38.166 --> 53:40.332 This came up eventually morning in the 53:40.332 --> 53:42.221 A I Ethics Council and uh we were 53:42.221 --> 53:41.919 having this discussion at lunch Bill , 53:41.929 --> 53:44.800 right ? What uh Craig was thinking and 53:44.810 --> 53:47.429 I think we need to comply with the US 53:47.439 --> 53:49.600 Constitution . I mean , we are a free 53:49.610 --> 53:51.832 society , we are a democratic country , 53:51.832 --> 53:53.888 we need to apply and comply with the 53:53.888 --> 53:55.999 laws . But at the same time , we need 53:55.999 --> 53:58.054 the mission capabilities to have the 53:58.054 --> 53:57.820 war fighting superiority . But I would 53:57.830 --> 54:01.449 let Bill as the primary dod 54:01.530 --> 54:03.820 policy leader in this space from A oh 54:03.830 --> 54:05.774 DS perspective , what , what would 54:05.774 --> 54:07.497 Secretary of Defense or Deputy 54:07.497 --> 54:09.608 Secretary of Defense think about this 54:09.608 --> 54:09.510 question . It's a very common question . 54:09.520 --> 54:11.919 Absolutely . So , I mean , I , you know , 54:11.929 --> 54:13.985 the , the A I we're interested in is 54:13.985 --> 54:15.985 responsible A I , right ? And , and 54:15.985 --> 54:17.762 that is embodied in the ethical 54:17.762 --> 54:19.985 principles , the dod uh has established 54:19.985 --> 54:22.159 and , and published uh it , there are 54:22.169 --> 54:24.679 five terms , reliability , equitable , 54:24.919 --> 54:27.879 uh governability , traceability , and 54:27.889 --> 54:30.350 responsibility . And you know , when 54:30.360 --> 54:32.582 you read those , those , those are five 54:32.582 --> 54:34.693 key terms that capture the dimensions 54:34.693 --> 54:37.320 around ethical use of A I . Um And , 54:37.330 --> 54:39.386 and when you read them , you realize 54:39.386 --> 54:41.219 they're really not meant for the 54:41.219 --> 54:41.110 machine , they're not meant for the A I , 54:41.120 --> 54:43.009 they're meant for the human . And 54:43.009 --> 54:45.231 that's because humans in the end are in 54:45.231 --> 54:47.120 control of how this technology is 54:47.120 --> 54:49.231 deployed and are responsible for it . 54:49.231 --> 54:51.231 So in the case of LLM is one of the 54:51.231 --> 54:53.453 first bits of guidance we've put out is 54:53.453 --> 54:55.453 that uh you are responsible for any 54:55.453 --> 54:58.080 action that is taken with them . So I , 54:58.239 --> 55:00.461 that is the guide , those are the guard 55:00.461 --> 55:02.572 rails that we as a society and as a , 55:02.572 --> 55:04.683 as a government as a , you know , dod 55:04.683 --> 55:06.906 defense organization are following in , 55:06.906 --> 55:08.906 in leveraging not only A I but also 55:08.906 --> 55:11.017 data and analytics because those also 55:11.017 --> 55:13.429 can be subjected or , or can uh can 55:13.439 --> 55:15.217 manifest problems with bias for 55:15.217 --> 55:17.899 instance , or uh lack of governability , 55:18.679 --> 55:21.149 I will say from a technical perspective , 55:21.159 --> 55:22.659 there's uh there have been 55:22.659 --> 55:25.310 demonstrations how when algorithms are , 55:25.320 --> 55:28.510 are uh developed robustly , uh they are 55:28.520 --> 55:30.520 actually better . Uh What I mean by 55:30.520 --> 55:32.409 that is when they're subjected to 55:32.409 --> 55:34.353 adversarial testing , they perform 55:34.353 --> 55:36.409 better , they generalize better . Um 55:36.409 --> 55:38.576 And so , although they might seem like 55:38.576 --> 55:41.030 um impediments to , you know , running 55:41.600 --> 55:44.300 fast into the A I fury if you will , 55:44.310 --> 55:46.477 they actually may be a very good thing 55:46.477 --> 55:48.477 for us because we'll understand the 55:48.477 --> 55:51.310 technology in a way , not only for , 55:51.320 --> 55:54.020 for our use , but maybe for what 55:54.030 --> 55:56.360 vulnerabilities there may be . So I 55:56.370 --> 55:59.780 think it's a plus and one of the areas 55:59.790 --> 56:01.623 from a ODNI perspective , we are 56:01.623 --> 56:03.734 looking is to establish a centralized 56:03.734 --> 56:05.512 governance that enables federal 56:05.512 --> 56:07.457 execution . So we need some way to 56:07.457 --> 56:09.401 govern these technologies for both 56:09.401 --> 56:11.512 human machine and machine , machine , 56:11.512 --> 56:13.790 teaming . And as with time , hopefully , 56:13.790 --> 56:15.901 we get more clarity probably sometime 56:15.901 --> 56:17.790 next year as we go about doing it 56:17.790 --> 56:19.790 because and there are certain other 56:19.790 --> 56:21.901 things too like law of armed conflict 56:21.901 --> 56:24.068 role of autonomous weapon systems , uh 56:24.068 --> 56:26.234 civil rights . So there are many areas 56:26.234 --> 56:28.290 how this is impacting . It's a major 56:28.290 --> 56:30.457 trend , it's a mega trend that will go 56:30.457 --> 56:30.360 on for the next 20 years . It will 56:30.370 --> 56:32.426 affect every single industry , every 56:32.426 --> 56:34.481 single function . So we have to be a 56:34.481 --> 56:36.703 bit more open minded and embrace at the 56:36.703 --> 56:38.703 same time , build capabilities that 56:38.703 --> 56:40.703 will give us strategic advantage to 56:40.703 --> 56:42.926 compete against any adversary anytime , 56:42.926 --> 56:44.926 right ? So I think it's a very good 56:44.926 --> 56:46.703 question and that's a important 56:46.703 --> 56:48.814 question for us as a nation . I think 56:48.814 --> 56:51.037 we are . You have any , I was gonna say 56:51.037 --> 56:51.020 one of the stories we , we are 56:51.030 --> 56:53.086 narratives we talk about at DPA . Is 56:53.086 --> 56:55.308 that seatbelts ? Actually let you drive 56:55.308 --> 56:57.419 faster . The kind of captures in like 56:57.419 --> 56:59.586 five words or whatever the , the , the 56:59.586 --> 57:01.863 story , the bill , the bill's response . 57:04.580 --> 57:06.524 I think we have two minutes . So I 57:06.524 --> 57:09.120 think we should wrap it up . Thank you 57:09.129 --> 57:11.479 all for listening . Thank you all for 57:11.489 --> 57:15.340 exciting doda . Maybe with that . Any 57:15.350 --> 57:18.429 closing remarks , anyone ? Yeah , a all 57:18.439 --> 57:20.495 I'd say is , you know , the programs 57:20.495 --> 57:22.828 that we have through R and E , you know , 57:22.828 --> 57:24.717 it's to democratize access to the 57:24.717 --> 57:26.828 capabilities that are critical for us 57:26.828 --> 57:28.772 to be successful here onshore with 57:28.772 --> 57:30.772 micro electronics . You know , it's 57:30.772 --> 57:32.550 gonna help bolt to the regional 57:32.550 --> 57:34.661 economies . It's gonna help develop a 57:34.661 --> 57:36.383 workforce uh to strengthen the 57:36.739 --> 57:38.572 microphonic ecosystem because my 57:38.572 --> 57:41.280 electronics uh underpins every system 57:41.290 --> 57:43.401 that you can think about , you know . 57:43.401 --> 57:45.457 So it's , it's very critical that us 57:45.457 --> 57:47.623 leadership in this uh across the cycle 57:47.623 --> 57:49.790 of design , manufacturing , assembly , 57:49.790 --> 57:51.901 packaging and testing . So that's the 57:51.901 --> 57:53.623 capability that I'm focused on 57:53.623 --> 57:55.846 obviously and will ensure that we'll uh 57:55.846 --> 57:57.957 onshore . A lot of those capabilities 57:57.957 --> 58:00.389 here . Any closing thoughts , Bill ? No , 58:00.399 --> 58:02.510 I appreciate the opportunity to be up 58:02.510 --> 58:04.732 here with the esteemed colleagues and I 58:04.732 --> 58:06.899 think these are exciting questions and 58:06.899 --> 58:09.066 I'm glad to be a part of it and uh you 58:09.066 --> 58:11.288 know , DARPA is always looking for that 58:11.288 --> 58:10.790 strategic surprise , strategic 58:10.800 --> 58:12.967 opportunities . If , if you think that 58:12.967 --> 58:14.911 there's something that DARPA isn't 58:14.911 --> 58:17.133 paying attention to that we should feel 58:17.133 --> 58:16.580 free to reach out . There's a way on 58:16.590 --> 58:18.812 the website . Also , Mars DARPA address 58:18.812 --> 58:20.757 is our first name dot last name at 58:20.757 --> 58:22.923 DARPA dot Mills . So you can crack the 58:22.923 --> 58:25.090 code . I'd be happy to hear from you . 58:25.090 --> 58:24.479 We're also always hiring program 58:24.489 --> 58:26.433 managers because everybody who has 58:26.433 --> 58:28.489 decision authority has an expiration 58:28.489 --> 58:30.600 date on their badge . So we're always 58:30.600 --> 58:32.656 kicking people out . So we're always 58:32.656 --> 58:34.711 having to bring new people in . So , 58:34.711 --> 58:34.560 you know , if DARPA is something that 58:34.580 --> 58:36.760 thinks it would be exciting for you , 58:36.929 --> 58:39.040 maybe not now , maybe sometime in the 58:39.040 --> 58:41.840 future . Keep it on your radar . Thank 58:41.850 --> 58:43.906 you all . I think we're on time . 20 58:43.906 --> 58:47.520 seconds left off to dug and th 58:59.320 --> 59:01.610 thank you Mr Menon and Panelist to 59:01.620 --> 59:03.509 conclude this year's conference . 59:03.509 --> 59:05.620 Please welcome back to the stage , Mr 59:05.620 --> 59:06.620 Doug Casa , 59:09.929 --> 59:13.830 close a all 59:13.840 --> 59:17.550 the doors and you out into the 59:17.889 --> 59:20.111 world . Hey , this is it closing time . 59:20.290 --> 59:22.346 Thank you , Ramesh . Thank you , our 59:22.346 --> 59:24.512 panelists on the emerging technologies 59:24.512 --> 59:26.179 panel and uh thank you A P in 59:26.179 --> 59:28.234 particular for saving me on asking a 59:28.234 --> 59:30.234 question . So and thank you all who 59:30.234 --> 59:32.457 have stuck with us for the remainder of 59:32.457 --> 59:34.401 the day , late into the day . Even 59:34.401 --> 59:34.320 though as I look out into the audience , 59:34.330 --> 59:36.497 most of you are from CIO and forced to 59:36.497 --> 59:38.663 be here but nonetheless , I appreciate 59:38.663 --> 59:40.719 it . So , and I wanted to start with 59:40.719 --> 59:42.750 thanking our amazing speakers uh 59:42.760 --> 59:44.982 throughout the past few days that we've 59:44.982 --> 59:47.149 had Lieutenant General Barrier PDD and 59:47.149 --> 59:49.371 I Stacy Dixon , General Kennedy Admiral 59:49.371 --> 59:51.260 Whitworth , General Henry General 59:51.260 --> 59:53.429 Gravey , honorable Sherman and Doctor 59:53.439 --> 59:55.661 Del Merritt and Laurie Wade . Uh , they 59:55.661 --> 59:57.550 all deserve a round of applause . 59:57.550 --> 59:59.439 They're really amazing speakers . 01:00:03.030 --> 01:00:05.086 I also want to thank Major Antwine , 01:00:05.086 --> 01:00:07.719 our , um , our speaker or I should say 01:00:07.729 --> 01:00:11.110 our um MC for this event for the past 01:00:11.120 --> 01:00:13.342 few days . That is a hard job . That is 01:00:13.342 --> 01:00:15.453 a very hard job . I did it one year . 01:00:15.453 --> 01:00:17.564 It was not fun . Not only do you have 01:00:17.564 --> 01:00:19.564 to pay attention the entire time uh 01:00:19.564 --> 01:00:21.676 because you don't know when someone's 01:00:21.676 --> 01:00:21.040 gonna end , you also have to pronounce 01:00:21.050 --> 01:00:22.828 names , which can be very nerve 01:00:22.828 --> 01:00:25.050 wracking . So , and a round of applause 01:00:25.050 --> 01:00:27.106 to you as well . Thank you so much . 01:00:29.840 --> 01:00:31.896 And then finally , I'd like to thank 01:00:31.896 --> 01:00:34.100 our cio uh communications team . They 01:00:34.110 --> 01:00:36.277 produced the video that you saw in the 01:00:36.277 --> 01:00:38.166 beginning when we had the kickoff 01:00:38.166 --> 01:00:40.332 session . And also I want to thank our 01:00:40.332 --> 01:00:40.010 office of corporate Communications . 01:00:40.020 --> 01:00:41.798 They've done a fantastic job of 01:00:41.798 --> 01:00:43.964 covering the event on social media and 01:00:43.964 --> 01:00:46.076 really helping us organize everything 01:00:46.076 --> 01:00:48.209 as well as NC si uh the who helped us 01:00:48.219 --> 01:00:50.441 really organize this conference and has 01:00:50.441 --> 01:00:52.441 done that every single year for the 01:00:52.441 --> 01:00:54.497 past 20 plus years . I want to thank 01:00:54.497 --> 01:00:56.608 all of them as well for the logistics 01:00:56.608 --> 01:00:56.060 in pulling this together . So thank you . 01:01:00.780 --> 01:01:03.679 So this , I was talking actually , this 01:01:03.689 --> 01:01:06.030 is my fifth notice conference . I was 01:01:06.040 --> 01:01:08.560 speaking with our former Executive 01:01:08.570 --> 01:01:11.300 Officer of Cio this morning . Uh Liz 01:01:11.310 --> 01:01:13.532 Doland , who's not with us here . She's 01:01:13.532 --> 01:01:15.588 monitoring from back in DC . But she 01:01:15.588 --> 01:01:17.810 reminded me this was my fifth . And she 01:01:17.810 --> 01:01:19.921 asked me , you know , I remember when 01:01:19.921 --> 01:01:22.032 we first met , we actually met at the 01:01:22.032 --> 01:01:21.979 Dous Conference . And do you remember 01:01:21.989 --> 01:01:24.129 your first uh talk that you gave on 01:01:24.139 --> 01:01:26.590 stage ? Uh And I said , yes , how could 01:01:26.600 --> 01:01:28.711 I forget that ? That was actually one 01:01:28.711 --> 01:01:30.878 of the most embarrassing moments of my 01:01:30.878 --> 01:01:32.989 public speaking life . Uh And many of 01:01:32.989 --> 01:01:35.100 you may might , may not remember this 01:01:35.100 --> 01:01:37.267 or may not have heard it . Um But when 01:01:37.267 --> 01:01:39.489 I was up on stage , I was talking about 01:01:39.489 --> 01:01:41.600 paradigms and paradigm shifts and how 01:01:41.600 --> 01:01:43.656 things that are now common knowledge 01:01:43.656 --> 01:01:45.656 today once seemed taboo . And I was 01:01:45.656 --> 01:01:47.822 speaking and I said the line and I got 01:01:47.822 --> 01:01:46.800 a little mixed up , but I said it 01:01:46.810 --> 01:01:48.977 confidently . So I didn't think anyone 01:01:48.977 --> 01:01:51.080 caught it . But I said the idea that 01:01:51.090 --> 01:01:53.239 the earth is flat is common knowledge 01:01:53.250 --> 01:01:56.959 today , which was once taboo back then . 01:01:58.209 --> 01:02:00.265 And I don't think anyone caught it , 01:02:00.265 --> 01:02:02.153 but Liz caught it and quite a few 01:02:02.153 --> 01:02:04.487 people actually caught it . Um But so I , 01:02:04.487 --> 01:02:06.598 I bring that up for two reasons . One 01:02:06.598 --> 01:02:08.431 because I want to set the record 01:02:08.431 --> 01:02:10.765 straight . I know that the world or the , 01:02:10.765 --> 01:02:12.876 the Earth is not flat . Uh You know , 01:02:12.876 --> 01:02:15.042 those of us that are shape experts and 01:02:15.042 --> 01:02:17.265 in the shape industry know that it is a 01:02:17.265 --> 01:02:19.989 uh an ellipsoid or an OML spheroid . Um 01:02:20.629 --> 01:02:22.685 You know , those , those are some of 01:02:22.685 --> 01:02:24.796 the best shapes , I think they're the 01:02:24.796 --> 01:02:26.851 same shape actually . Uh But they're 01:02:26.851 --> 01:02:26.659 definitely the shape of the earth . So 01:02:26.669 --> 01:02:28.780 I want to get that on the record . Uh 01:02:28.780 --> 01:02:30.947 But I also brought it up because , you 01:02:30.947 --> 01:02:32.947 know , the the concept , you know , 01:02:32.947 --> 01:02:35.189 crazy concepts uh that are commonplace 01:02:35.199 --> 01:02:37.088 today . Actually , we we've heard 01:02:37.088 --> 01:02:39.310 throughout the entire week , you know , 01:02:39.310 --> 01:02:41.477 open source intelligence , I saw quite 01:02:41.477 --> 01:02:43.532 a few presentations in the Expo Hall 01:02:43.532 --> 01:02:45.588 and had quite a few briefings on the 01:02:45.588 --> 01:02:47.699 value of open source . And I will say 01:02:47.699 --> 01:02:49.755 20 years ago when I was a collection 01:02:49.755 --> 01:02:49.050 requirements manager in the 01:02:49.060 --> 01:02:51.227 intelligence community , I remember me 01:02:51.227 --> 01:02:53.489 mentioning the term cent and my boss 01:02:53.500 --> 01:02:55.667 corrected me saying that that's not an 01:02:55.667 --> 01:02:57.667 intelligence discipline . There are 01:02:57.667 --> 01:02:59.833 only four cent is not one of them . So 01:02:59.833 --> 01:03:01.944 don't call it cent , it's called open 01:03:01.944 --> 01:03:04.290 source . Well , today , open source OIN , 01:03:04.300 --> 01:03:06.244 which is formally recognized as an 01:03:06.244 --> 01:03:08.411 intelligence discipline is not a taboo 01:03:08.411 --> 01:03:10.633 concept . It's actually a line of force 01:03:10.633 --> 01:03:12.744 first resort and knowing what's going 01:03:12.744 --> 01:03:14.800 on in the ground and what's going on 01:03:14.800 --> 01:03:16.856 around the world . Also , we've seen 01:03:16.856 --> 01:03:18.967 quite a few themes which are paradigm 01:03:18.967 --> 01:03:21.022 shifts for us today . So as we think 01:03:21.022 --> 01:03:23.290 about operating , you know , we talked 01:03:23.300 --> 01:03:25.522 a lot about data centers and circuits . 01:03:26.040 --> 01:03:28.550 Not too long ago , we were operating 01:03:28.560 --> 01:03:30.671 everything out of our data centers in 01:03:30.671 --> 01:03:32.820 Washington DC . You heard a lot today 01:03:32.830 --> 01:03:35.060 on the need for not only thinking about 01:03:35.070 --> 01:03:37.070 alternate communications , separate 01:03:37.070 --> 01:03:38.570 from terrestrial satellite 01:03:38.570 --> 01:03:40.737 communications , but how do we operate 01:03:40.737 --> 01:03:42.681 in denied environments ? How do we 01:03:42.681 --> 01:03:44.848 operate in disconnected environments ? 01:03:44.848 --> 01:03:46.848 Those are concepts that we've never 01:03:46.848 --> 01:03:48.959 talked about before over the past few 01:03:48.959 --> 01:03:51.126 years . And we really are prioritizing 01:03:51.126 --> 01:03:53.292 today . Cyber is another , another big 01:03:53.292 --> 01:03:55.514 topic that's been a paradigm shift . In 01:03:55.514 --> 01:03:55.290 fact , I was speaking with Honorable 01:03:55.300 --> 01:03:57.411 Sherman yesterday and we were talking 01:03:57.411 --> 01:03:59.356 about one of the things we're most 01:03:59.356 --> 01:04:01.522 proud of in Dia Cio over the past year 01:04:01.522 --> 01:04:03.744 is passing our cyber inspection that it 01:04:03.744 --> 01:04:05.744 was the hardest thing to do . And I 01:04:05.744 --> 01:04:07.911 will say we were the first agency that 01:04:07.911 --> 01:04:10.139 passed both the new cyber inspection 01:04:10.149 --> 01:04:12.316 from JF HQ Doen on Nipper and zipper . 01:04:12.316 --> 01:04:14.482 It was quite an accomplishment , but I 01:04:14.482 --> 01:04:16.593 told him that changed our culture and 01:04:16.593 --> 01:04:18.820 it changed in many ways , not just in , 01:04:18.830 --> 01:04:21.052 in terms of stressing the importance of 01:04:21.052 --> 01:04:23.108 technology refresh , so we can patch 01:04:23.108 --> 01:04:25.052 equipment so we can do things like 01:04:25.052 --> 01:04:27.108 comply to connect foundation of zero 01:04:27.108 --> 01:04:29.108 trust , but also simple things like 01:04:29.108 --> 01:04:31.219 having a change management process as 01:04:31.219 --> 01:04:33.219 we change the infrastructure of our 01:04:33.219 --> 01:04:35.052 network , making sure that we're 01:04:35.052 --> 01:04:37.274 actually making smart decisions . We're 01:04:37.274 --> 01:04:36.850 not doing that in the middle of a 01:04:36.860 --> 01:04:38.860 crisis that we're actually tracking 01:04:38.860 --> 01:04:41.138 every step of the way what we're doing , 01:04:41.138 --> 01:04:43.027 who's changing things . So we can 01:04:43.027 --> 01:04:45.249 actually manage everything . Our entire 01:04:45.249 --> 01:04:47.360 architecture is an enterprise . Those 01:04:47.360 --> 01:04:49.360 were foreign concepts and still are 01:04:49.360 --> 01:04:51.527 foreign concepts to many elements that 01:04:51.527 --> 01:04:53.527 operate information technology also 01:04:53.527 --> 01:04:55.527 entitlements , right ? If you think 01:04:55.527 --> 01:04:57.582 about the intelligence community and 01:04:57.582 --> 01:04:59.749 where we've been , what is the biggest 01:04:59.749 --> 01:05:01.916 threat ? And I will say when I started 01:05:01.916 --> 01:05:03.916 this job , someone asked me what my 01:05:03.916 --> 01:05:05.916 biggest concern was and I said it's 01:05:05.916 --> 01:05:08.082 cyber , it's having a compromise and I 01:05:08.082 --> 01:05:10.304 was told that doesn't happen in the top 01:05:10.304 --> 01:05:12.304 secret world . Well , guess what it 01:05:12.304 --> 01:05:14.304 does and you know what particularly 01:05:14.304 --> 01:05:16.304 happens , which we've seen over and 01:05:16.304 --> 01:05:18.304 over are insider threats . That's a 01:05:18.304 --> 01:05:20.527 real threat . And that directly relates 01:05:20.527 --> 01:05:19.479 to a number of the concepts . We've 01:05:19.489 --> 01:05:21.656 talked this week especially zero trust 01:05:21.656 --> 01:05:23.489 principles like entitlements and 01:05:23.489 --> 01:05:25.433 getting identities down . And that 01:05:25.433 --> 01:05:27.433 brings me to another topic which is 01:05:27.433 --> 01:05:29.656 taboo or has been taboo traditionally , 01:05:29.656 --> 01:05:31.933 which is sharing with foreign partners . 01:05:32.080 --> 01:05:34.302 This was the first five I panel that we 01:05:34.302 --> 01:05:36.524 ever had as a do at a dota conference , 01:05:36.524 --> 01:05:38.691 believe it or not when we developed it 01:05:38.691 --> 01:05:41.100 even five years ago , the concept was 01:05:41.110 --> 01:05:43.649 as we do no form first , that has 01:05:43.659 --> 01:05:45.826 dramatically changed in the design and 01:05:45.826 --> 01:05:48.103 implementation of technology . In fact , 01:05:48.103 --> 01:05:50.326 everything that dia is developing today 01:05:50.326 --> 01:05:52.879 is five I first and looking at our 01:05:52.889 --> 01:05:54.722 collaboration with international 01:05:54.722 --> 01:05:56.500 partners , both traditional and 01:05:56.500 --> 01:05:58.556 nontraditional things that we've now 01:05:58.556 --> 01:06:00.167 signed up for , to create in 01:06:00.167 --> 01:06:02.389 partnership with other agencies are now 01:06:02.389 --> 01:06:01.639 becoming commonplace in our 01:06:01.649 --> 01:06:04.310 architecture . So we've heard those 01:06:04.320 --> 01:06:06.487 themes this week . But there's another 01:06:06.487 --> 01:06:08.764 theme that we heard during the service . 01:06:08.764 --> 01:06:11.729 CIO panel , Jane mentioned the quote . 01:06:11.969 --> 01:06:14.600 She's just the cio I will tell you 01:06:14.610 --> 01:06:16.709 that's a paradigm shift when I think 01:06:16.719 --> 01:06:18.552 about everything that we've been 01:06:18.552 --> 01:06:20.719 through . And I mentioned on the first 01:06:20.719 --> 01:06:22.830 day Afghanistan , who is the last out 01:06:22.830 --> 01:06:25.909 of Afghanistan , who's the cio when we 01:06:25.919 --> 01:06:28.252 think about Ukraine and Russia conflict , 01:06:28.252 --> 01:06:31.000 who was the first in , it was just the 01:06:31.010 --> 01:06:33.489 CIO when we think about where we are 01:06:33.500 --> 01:06:35.722 right now with Israel and Hamas in that 01:06:35.722 --> 01:06:38.580 conflict , who was the first called , 01:06:38.860 --> 01:06:40.971 it was just the CIO to help with that 01:06:40.971 --> 01:06:43.489 communication . And one of the biggest 01:06:43.500 --> 01:06:45.444 paradigm shifts that we've seen in 01:06:45.444 --> 01:06:47.500 every one of those conflicts is it's 01:06:47.500 --> 01:06:49.111 not a kinetic war like we're 01:06:49.111 --> 01:06:51.056 traditionally used to . What's the 01:06:51.056 --> 01:06:53.278 first thing that happens ? It's a cyber 01:06:53.278 --> 01:06:55.111 incident . It's disrupted in our 01:06:55.111 --> 01:06:57.333 command and control and our command and 01:06:57.333 --> 01:06:59.556 control is everything that we've talked 01:06:59.556 --> 01:07:01.722 about at this conference . I mentioned 01:07:01.722 --> 01:07:03.833 during one of the panels , everything 01:07:03.833 --> 01:07:03.419 fails at interface . Well , you know , 01:07:03.429 --> 01:07:05.485 what keeps interfaces flowing , it's 01:07:05.485 --> 01:07:07.707 technology and that comes from just the 01:07:07.707 --> 01:07:09.820 CIO . But the final thing that I want 01:07:09.830 --> 01:07:12.709 to mention in this , we talked a lot 01:07:12.719 --> 01:07:14.941 about our career advice and some of the 01:07:14.941 --> 01:07:16.997 best things that we've learned , you 01:07:16.997 --> 01:07:19.108 know , as seniors in this community . 01:07:19.108 --> 01:07:21.108 And again , we have a lot of up and 01:07:21.108 --> 01:07:22.886 coming folks within CIO in this 01:07:22.886 --> 01:07:22.729 audience because they're forced to be 01:07:22.739 --> 01:07:25.750 here . But one of the best pieces of 01:07:25.760 --> 01:07:28.469 advice I can give you as a cio of 01:07:28.479 --> 01:07:30.535 defense intelligence agency is trust 01:07:30.535 --> 01:07:32.423 your people , right ? Some of the 01:07:32.423 --> 01:07:34.590 hardest problems that we've had do not 01:07:34.590 --> 01:07:36.590 come from myself . They do not come 01:07:36.590 --> 01:07:38.812 from ep as the deputy cio , they do not 01:07:38.812 --> 01:07:40.979 come from our other leaders within our 01:07:40.979 --> 01:07:42.979 organization . They come from those 01:07:42.979 --> 01:07:42.669 working the problems day in and day out . 01:07:42.679 --> 01:07:44.790 And that is the number one thing that 01:07:44.790 --> 01:07:46.846 I've learned as a leader is to trust 01:07:46.846 --> 01:07:49.068 the folks that know the work , know the 01:07:49.068 --> 01:07:51.290 technology and have the expertise . And 01:07:51.290 --> 01:07:52.957 speaking of this , one of the 01:07:52.957 --> 01:07:55.068 traditions of this dous conference in 01:07:55.068 --> 01:07:57.235 closing this out is to watch a closing 01:07:57.235 --> 01:07:59.629 video . This is the first time that D I 01:07:59.639 --> 01:08:03.370 cio has created this video . We're 01:08:03.379 --> 01:08:05.490 about to watch it . I have never seen 01:08:05.490 --> 01:08:08.530 this video because I trust our team 01:08:09.389 --> 01:08:11.445 and I trust that they've done a good 01:08:11.445 --> 01:08:13.445 job and I know this is gonna be the 01:08:13.445 --> 01:08:16.229 perfect closing to this conference . So 01:08:16.240 --> 01:08:18.950 without further ado , let's roll that 01:08:18.959 --> 01:08:19.450 video . 01:08:27.669 --> 01:08:30.609 This is our first time in Portland and 01:08:30.620 --> 01:08:32.879 this is our 20th conference . I counted 01:08:42.399 --> 01:08:44.566 the technology that I was most excited 01:08:44.566 --> 01:08:47.819 to hear about was A I and they didn't 01:08:47.830 --> 01:08:50.459 fail . Uh I was most excited to hear 01:08:50.470 --> 01:08:52.970 about new technologies in artificial 01:08:52.979 --> 01:08:55.569 intelligence and data management . You 01:08:55.580 --> 01:08:57.691 know , two critical capabilities that 01:08:57.691 --> 01:08:59.913 are really foundational to our programs 01:08:59.913 --> 01:09:02.024 at DAA . I was really excited to hear 01:09:02.024 --> 01:09:05.020 about data . Um As somebody who manages 01:09:05.104 --> 01:09:07.185 recruitment , retention and training 01:09:07.194 --> 01:09:09.527 and development for the it career field , 01:09:09.527 --> 01:09:12.165 I really struggle with being able to 01:09:12.174 --> 01:09:14.007 evaluate the effectiveness of my 01:09:14.007 --> 01:09:16.118 programs . The things that excited me 01:09:16.145 --> 01:09:18.345 are really the things Senator Brown , 01:09:18.354 --> 01:09:20.604 uh artificial intelligence , machine 01:09:20.615 --> 01:09:24.220 learning and quantum capabilities . We 01:09:24.229 --> 01:09:25.951 realize that technology is the 01:09:25.951 --> 01:09:28.350 foundation of how we're gonna survive , 01:09:28.770 --> 01:09:31.160 how we're gonna enable intelligence and 01:09:31.169 --> 01:09:33.225 how we're gonna fight the next war . 01:09:33.225 --> 01:09:36.660 The Cyber domain is interacting with 01:09:36.669 --> 01:09:38.669 the adversary every day , all day . 01:09:38.669 --> 01:09:40.780 Right now , it's not a future thing . 01:09:40.780 --> 01:09:43.002 They are interacting with the adversary 01:09:43.002 --> 01:09:45.225 every day , all day . 20 years ago . It 01:09:45.225 --> 01:09:47.447 actually used to be what was considered 01:09:47.447 --> 01:09:49.613 sort of a basement function within our 01:09:49.613 --> 01:09:51.613 community . Now , it's on the front 01:09:51.613 --> 01:09:53.336 line . Technology is the first 01:09:53.336 --> 01:09:55.113 capability in and it's the last 01:09:55.113 --> 01:09:57.113 capability out every crisis that we 01:09:57.113 --> 01:09:59.336 support adversary disruption . Where is 01:09:59.336 --> 01:10:01.502 the adversary trying to go ? How could 01:10:01.502 --> 01:10:00.899 the adversary disrupt ? So we don't 01:10:00.910 --> 01:10:03.021 have a single point of failure in our 01:10:03.021 --> 01:10:06.450 enterprise . We are at a pivotal moment 01:10:06.459 --> 01:10:08.070 in the history of humanity . 01:10:08.140 --> 01:10:09.751 Technologies like artificial 01:10:09.751 --> 01:10:11.899 intelligence is not just impacting P 01:10:11.910 --> 01:10:14.330 it's impacting every single industry on 01:10:14.339 --> 01:10:17.069 the planet . Technology is the market 01:10:17.080 --> 01:10:20.359 differentiator between the IC being in 01:10:20.370 --> 01:10:23.490 the intelligence business versus being 01:10:23.500 --> 01:10:26.319 in the history reporting business . We 01:10:26.330 --> 01:10:30.229 move at the speed of trust and that's 01:10:30.240 --> 01:10:32.529 where partnerships start . If we want 01:10:32.540 --> 01:10:35.549 real partners , we've got to invest in 01:10:35.560 --> 01:10:37.569 that partnership . So as we look at 01:10:37.580 --> 01:10:39.524 especially the challenges over the 01:10:39.524 --> 01:10:41.747 course of this past year , every single 01:10:41.747 --> 01:10:43.580 one of those has depended on our 01:10:43.580 --> 01:10:45.469 relationships with our partners . 01:10:45.469 --> 01:10:47.636 Partnerships are critical , especially 01:10:47.636 --> 01:10:49.524 for us on the JW program , we are 01:10:49.524 --> 01:10:51.358 constantly looking at how we can 01:10:51.358 --> 01:10:53.939 partner with industry on delivering 01:10:53.950 --> 01:10:56.950 services , bringing in new technologies , 01:10:57.089 --> 01:10:59.311 uh changing the way that we do business 01:10:59.311 --> 01:11:01.311 that makes our delivery of services 01:11:01.311 --> 01:11:03.533 more efficient . The only way that we , 01:11:03.533 --> 01:11:05.756 that we're going to continue to protect 01:11:05.756 --> 01:11:07.811 our democracy is by working together 01:11:07.811 --> 01:11:10.950 because it affects us all . It's no 01:11:10.959 --> 01:11:14.479 longer um sin for sin or human , for 01:11:14.490 --> 01:11:16.212 human . It's really all source 01:11:16.220 --> 01:11:18.669 intelligence and that's the the basis 01:11:18.759 --> 01:11:21.770 of what dia is all about . They give us 01:11:21.779 --> 01:11:23.946 an opportunity to , to be able to meet 01:11:23.946 --> 01:11:25.723 with the military um and senior 01:11:25.723 --> 01:11:27.835 enlisted leaders that are pretty much 01:11:27.835 --> 01:11:29.979 located throughout the world . And uh 01:11:30.000 --> 01:11:32.167 this , this became a perfect venue for 01:11:32.167 --> 01:11:34.389 all them to come together and for us to 01:11:34.389 --> 01:11:36.333 gather and definitely come up with 01:11:36.333 --> 01:11:38.444 different procedures of how to better 01:11:38.444 --> 01:11:40.278 serve mission . The adversary is 01:11:40.278 --> 01:11:43.430 getting smarter on how they intend 01:11:43.939 --> 01:11:47.040 to attack our technical capabilities . 01:11:47.310 --> 01:11:50.290 And so we need to stay a step ahead . 01:11:50.299 --> 01:11:52.355 Um And we're not gonna be able to do 01:11:52.355 --> 01:11:54.355 that unless we work collaboratively 01:11:54.355 --> 01:11:56.577 together . You know , dos is not just a 01:11:56.577 --> 01:11:59.020 three day annual conference , it's 01:11:59.029 --> 01:12:01.640 about opportunity , it's about creating 01:12:01.649 --> 01:12:04.200 pathways , it's about conversations , 01:12:04.500 --> 01:12:08.200 any crisis around the world . It is not 01:12:08.209 --> 01:12:10.390 one country versus another . It is the 01:12:10.430 --> 01:12:13.729 totality of multiple countries . So our 01:12:13.740 --> 01:12:17.000 ability to form partnerships is key 01:12:17.270 --> 01:12:20.520 to enabling us to move from chaos to 01:12:20.529 --> 01:12:23.859 clear time is of the essence . 01:12:24.830 --> 01:12:28.220 So if we don't solve problems , uh 01:12:28.229 --> 01:12:31.529 now , then we will be unable 01:12:31.750 --> 01:12:34.919 to engage in a way that allows us to be 01:12:35.189 --> 01:12:37.629 competitive in the future . We are 01:12:37.640 --> 01:12:39.529 learning from each other . We are 01:12:39.529 --> 01:12:41.089 sharing our information and 01:12:41.100 --> 01:12:44.680 collectively , we uphold the true 01:12:44.689 --> 01:12:48.569 democratic values of free world . We 01:12:48.580 --> 01:12:51.870 cannot wait to uh uh enable the 01:12:51.879 --> 01:12:54.779 technological solutions until after 01:12:54.790 --> 01:12:56.901 that risk , appetite change . We have 01:12:56.901 --> 01:12:59.012 to do that in advance . There will be 01:12:59.012 --> 01:13:01.234 people who will look at our country and 01:13:01.234 --> 01:13:00.830 say , well , let's figure out how we 01:13:00.839 --> 01:13:02.561 can distract them , how we can 01:13:02.561 --> 01:13:04.783 interrupt them so that they aren't able 01:13:04.783 --> 01:13:06.672 to prepare and be what the United 01:13:06.672 --> 01:13:08.783 States normally would be in a time of 01:13:08.783 --> 01:13:10.783 conflict . Wow , dous conference in 01:13:10.783 --> 01:13:12.950 just one word . I think it's got to be 01:13:12.950 --> 01:13:15.189 opportunity , fascinating , diverse 01:13:15.200 --> 01:13:19.200 synergy , phenomenal . We 01:13:19.209 --> 01:13:21.720 are swimming in a world of data looking 01:13:21.729 --> 01:13:24.709 for answers and within the IC , what we 01:13:24.720 --> 01:13:27.470 do is actually add clarity to the chaos 01:13:27.479 --> 01:13:28.757 that's all around us . 01:13:41.970 --> 01:13:43.970 Well , that was really awesome . Uh 01:13:43.970 --> 01:13:45.970 Thank you . And it turns out we can 01:13:45.970 --> 01:13:48.026 trust our communications team within 01:13:48.026 --> 01:13:50.303 CIO . No . Great job again . Thank you . 01:13:52.950 --> 01:13:55.680 All right . As we close out our 2023 01:13:55.689 --> 01:13:58.000 dous worldwide conference , I'm gonna 01:13:58.009 --> 01:14:01.770 end my remarks with a prepared speech 01:14:01.779 --> 01:14:04.209 written by Liz Doland over in our 01:14:04.220 --> 01:14:06.387 Office of Strategic Communications . I 01:14:06.387 --> 01:14:08.609 asked her , you know , you help prepare 01:14:08.609 --> 01:14:10.609 my first one . Maybe you could help 01:14:10.609 --> 01:14:13.609 prepare this 15 years later , I don't 01:14:13.620 --> 01:14:16.669 know , but I'm , I'm gonna read it and 01:14:16.680 --> 01:14:19.029 um there are , I'm gonna share six 01:14:19.040 --> 01:14:20.873 major takeaways from this year's 01:14:20.873 --> 01:14:22.979 conference with a bit of holiday . 01:14:22.990 --> 01:14:26.520 Cheer . So during this year's dous 01:14:26.529 --> 01:14:29.660 worldwide conference , we received six 01:14:29.669 --> 01:14:32.220 plenary panels panels , five eye 01:14:32.229 --> 01:14:34.930 partners , four lieutenant generals , 01:14:35.169 --> 01:14:38.049 three days of fun , two days of 01:14:38.060 --> 01:14:40.870 breakouts and the largest it conference 01:14:40.879 --> 01:14:43.970 in the IC . So thank you for joining us . 01:14:44.410 --> 01:14:47.649 Thank you . Look forward to our next 01:14:47.660 --> 01:14:50.209 conference in 2024 again at the United 01:14:50.220 --> 01:14:52.220 States Strategic Command in Omaha , 01:14:52.220 --> 01:14:54.387 Nebraska , which is always a good time 01:14:54.387 --> 01:14:56.553 and always a large audience given that 01:14:56.553 --> 01:14:58.442 we uh have that location with the 01:14:58.442 --> 01:15:00.609 command and many academia and industry 01:15:00.609 --> 01:15:02.831 partners that are in that area and look 01:15:02.831 --> 01:15:05.330 forward to uh a successful conference 01:15:05.339 --> 01:15:07.339 next year and I hope you all have a 01:15:07.339 --> 01:15:09.810 safe and happy holiday season again . 01:15:09.910 --> 01:15:12.077 Thank you for joining us and we'll see 01:15:12.077 --> 01:15:11.770 you next year .