1 00:00:06,410 --> 00:00:08,110 - So good morning. 2 00:00:08,110 --> 00:00:10,230 We'll go ahead and get started. 3 00:00:10,230 --> 00:00:12,110 I'd like to say thank you for your attendance this morning, 4 00:00:12,110 --> 00:00:14,174 8:30 a.m. on day three is, 5 00:00:14,174 --> 00:00:15,529 it's almost heroic 6 00:00:15,529 --> 00:00:17,720 (audience laughing) in some circles. 7 00:00:17,720 --> 00:00:19,090 So thank you so much for being part 8 00:00:19,090 --> 00:00:20,630 of our panel this morning. 9 00:00:20,630 --> 00:00:22,320 We've got a... (cell phone chimes) 10 00:00:22,320 --> 00:00:23,433 Wow, is that me? 11 00:00:26,440 --> 00:00:27,273 It's not. 12 00:00:27,273 --> 00:00:28,260 All right. 13 00:00:28,260 --> 00:00:30,410 So maybe that's my opening joke. 14 00:00:30,410 --> 00:00:31,243 You know, it's interesting. 15 00:00:31,243 --> 00:00:32,076 Earlier today I was thinking, 16 00:00:32,076 --> 00:00:33,660 well, to be a good panel moderator, 17 00:00:33,660 --> 00:00:36,290 I probably oughta try to make people laugh a little bit. 18 00:00:36,290 --> 00:00:39,780 So then I Googled "Panel Moderator Jokes". 19 00:00:39,780 --> 00:00:40,613 (audience laughing) 20 00:00:40,613 --> 00:00:42,280 And I didn't really find anything, 21 00:00:42,280 --> 00:00:43,854 so I'm sorry. 22 00:00:43,854 --> 00:00:45,450 (audience laughing) You know? 23 00:00:45,450 --> 00:00:47,370 What I did find is I found a bunch of tips 24 00:00:47,370 --> 00:00:50,720 on how to be a good panel moderator. 25 00:00:50,720 --> 00:00:52,910 Sadly, I found them while I was standing here at the podium, 26 00:00:52,910 --> 00:00:53,743 so I don't know 27 00:00:53,743 --> 00:00:55,080 how much use they are to me. (audience laughing) 28 00:00:55,080 --> 00:00:56,930 So let's go ahead and get started. 29 00:00:56,930 --> 00:00:58,440 I'm kinda a last minute kinda guy, 30 00:00:58,440 --> 00:00:59,520 I like to live in the moment, 31 00:00:59,520 --> 00:01:01,010 so, as you can see, 32 00:01:01,010 --> 00:01:02,723 so none of this was planned. 33 00:01:02,723 --> 00:01:05,030 I'll introduce our panel members very briefly. 34 00:01:05,030 --> 00:01:06,410 I'll give them some time to talk, 35 00:01:06,410 --> 00:01:09,140 and then the intent here is to have brief comments 36 00:01:09,140 --> 00:01:10,360 around a set of questions 37 00:01:10,360 --> 00:01:12,970 and really have an interactive session. 38 00:01:12,970 --> 00:01:14,070 So we endeavor to do that, 39 00:01:14,070 --> 00:01:15,920 so we'll manage our time accordingly. 40 00:01:17,893 --> 00:01:19,550 So we have Michael Conlin in the center. 41 00:01:19,550 --> 00:01:22,537 Michael Conlin is the Chief Data Officer for DOD. 42 00:01:23,610 --> 00:01:27,620 He's under the Office of Under Secretary of Defense, 43 00:01:27,620 --> 00:01:31,300 Office of the Chief Management Office, 44 00:01:31,300 --> 00:01:34,780 and comes to us with many, many years of experience, 45 00:01:34,780 --> 00:01:36,620 20 plus years of experience. 46 00:01:36,620 --> 00:01:41,540 He declares himself as a "Serial Start Up Entrepreneur", 47 00:01:41,540 --> 00:01:44,240 or "Serial Entrepreneur", I should say, 48 00:01:44,240 --> 00:01:47,270 has served in many positions across the globe 49 00:01:47,270 --> 00:01:48,973 in five different continents, 50 00:01:50,540 --> 00:01:52,670 many, many high-tech start ups, et cetera. 51 00:01:52,670 --> 00:01:55,300 So Michael is very, very interesting. 52 00:01:55,300 --> 00:01:58,770 He's got some very interesting personal pursuits 53 00:01:58,770 --> 00:02:00,160 in his bio. 54 00:02:00,160 --> 00:02:03,400 Some of which include blacksmith, cook, enamelist, 55 00:02:03,400 --> 00:02:07,600 gaffer, goldsmith, lamp worker, lapidary, and poet. 56 00:02:07,600 --> 00:02:09,470 And if you're like me, 57 00:02:09,470 --> 00:02:12,490 and I'll self admit that I'm probably not the most 58 00:02:12,490 --> 00:02:16,210 fluent in some of the lesser-used English terms, 59 00:02:16,210 --> 00:02:18,365 I did have to look up the word "lapidary". 60 00:02:18,365 --> 00:02:19,940 (audience laughing) So. 61 00:02:19,940 --> 00:02:21,050 And then we have Nancy Morgan. 62 00:02:21,050 --> 00:02:23,890 Nancy Morgan is the IC CDO, 63 00:02:23,890 --> 00:02:27,480 recently appointed in May of 2019. 64 00:02:27,480 --> 00:02:29,150 Prior to Nancy was Steve Prosser, 65 00:02:29,150 --> 00:02:32,340 and she comes to us with many years of experience, 66 00:02:32,340 --> 00:02:35,669 25 plus years of experience as a CIA Officer, 67 00:02:35,669 --> 00:02:40,290 and has been involved in numerous ICY Flagship Programs 68 00:02:40,290 --> 00:02:41,940 where she's been very influential 69 00:02:42,779 --> 00:02:45,840 in shaping today's state of Eyesight. 70 00:02:45,840 --> 00:02:47,380 So we look forward to her leadership, 71 00:02:47,380 --> 00:02:51,623 and she has been fantastic joining the IC CDO Council, 72 00:02:53,170 --> 00:02:54,003 and leading us, 73 00:02:54,003 --> 00:02:58,773 and continuing to drive us to make momentous changes 74 00:02:58,773 --> 00:03:01,970 in the area of of Data Integration across the IC. 75 00:03:01,970 --> 00:03:03,220 And as you'll see today, 76 00:03:03,220 --> 00:03:05,859 great partnership with DOD as well. 77 00:03:05,859 --> 00:03:10,859 Nancy's fun fact is she learned to sail on Lake Michigan. 78 00:03:11,230 --> 00:03:12,430 So what's fun about that? 79 00:03:12,430 --> 00:03:16,094 Imagine the temperature if you tip the sailboat over. 80 00:03:16,094 --> 00:03:17,413 - I did. - So I, 81 00:03:17,413 --> 00:03:20,100 I have an experience of doing that in Florida. 82 00:03:20,100 --> 00:03:21,630 Thankfully it was not Lake Michigan, 83 00:03:21,630 --> 00:03:23,480 and I wound up stuck in the water for a while 84 00:03:23,480 --> 00:03:26,530 because I couldn't get the catamaran back upright. 85 00:03:26,530 --> 00:03:28,800 And then we have John Turner. 86 00:03:28,800 --> 00:03:29,720 John Turner comes to us 87 00:03:29,720 --> 00:03:31,443 from the Office of Under Secretary Defense 88 00:03:31,443 --> 00:03:35,230 under the Director of ISR Enterprise Capabilities. 89 00:03:35,230 --> 00:03:37,140 12 years in the IC, 90 00:03:37,140 --> 00:03:40,610 and he's been invaluable as the integration glue 91 00:03:40,610 --> 00:03:42,470 between DOD and IC, 92 00:03:42,470 --> 00:03:44,893 and a great team member to work with. 93 00:03:45,730 --> 00:03:48,647 John's fun fact is he doesn't have a fun fact. 94 00:03:48,647 --> 00:03:49,480 (audience laughing) So if you have 95 00:03:49,480 --> 00:03:51,370 an opportunity to talk to him after the session, 96 00:03:51,370 --> 00:03:52,360 I would challenge him on that, 97 00:03:52,360 --> 00:03:54,380 'cause everyone's gotta have a fun fact. 98 00:03:54,380 --> 00:03:55,213 - Anti fun fact. - So. 99 00:03:55,213 --> 00:03:56,990 All right, well I'll pause now, 100 00:03:56,990 --> 00:03:59,710 and I'll give each panel member a moment to share 101 00:03:59,710 --> 00:04:00,770 some opening comments, 102 00:04:00,770 --> 00:04:02,490 and why don't we start with you, Nancy? 103 00:04:02,490 --> 00:04:03,440 - Sure. 104 00:04:03,440 --> 00:04:04,450 Good morning. 105 00:04:04,450 --> 00:04:07,360 Happy to be here today to discuss how the IC is taking 106 00:04:07,360 --> 00:04:10,610 critical steps to support the increasingly data driven 107 00:04:10,610 --> 00:04:13,550 services and capabilities that are required to achieve 108 00:04:13,550 --> 00:04:15,710 our national security mission. 109 00:04:15,710 --> 00:04:17,150 Before I continue, I must say, 110 00:04:17,150 --> 00:04:18,260 as that native Chicagoan, 111 00:04:18,260 --> 00:04:19,340 I find this level of heat 112 00:04:19,340 --> 00:04:21,330 and humidity here a little bit daunting. 113 00:04:21,330 --> 00:04:22,742 Little bit humid here. 114 00:04:22,742 --> 00:04:25,340 I've only been in my position for about three months, 115 00:04:25,340 --> 00:04:27,820 but I'm excited and energized by all the great work 116 00:04:27,820 --> 00:04:29,530 that we have going on in the IC 117 00:04:29,530 --> 00:04:31,540 and in partnership with DOD 118 00:04:31,540 --> 00:04:34,120 to use tools and data in new ways to increase 119 00:04:34,120 --> 00:04:36,920 our intelligence integration and collaboration. 120 00:04:36,920 --> 00:04:39,640 Including how that helps us build resiliency 121 00:04:39,640 --> 00:04:42,570 and strengthen our cyber security posture. 122 00:04:42,570 --> 00:04:43,860 In 2017, 123 00:04:43,860 --> 00:04:44,989 we published the first 124 00:04:44,989 --> 00:04:49,770 IC Information Environment Data Strategy, 2017 to '21. 125 00:04:49,770 --> 00:04:52,270 There's a couple copies here on the table at the end. 126 00:04:52,270 --> 00:04:53,850 But also in the Digital Age, 127 00:04:53,850 --> 00:04:55,210 thanks to my colleague over here, 128 00:04:55,210 --> 00:04:56,050 you can go to the back 129 00:04:56,050 --> 00:04:58,760 and just scan with the QR code on your phone. 130 00:04:58,760 --> 00:04:59,940 So welcome to the Digital Age. 131 00:04:59,940 --> 00:05:01,290 Little bit strange coming from CIA 132 00:05:01,290 --> 00:05:02,630 to be able to do that so easily 133 00:05:02,630 --> 00:05:03,840 here these days. (audience laughing) 134 00:05:03,840 --> 00:05:04,673 I know, 135 00:05:04,673 --> 00:05:07,000 we're not quite used to being quite so open out there. 136 00:05:07,000 --> 00:05:09,440 This strategy was developed through a very collaborative 137 00:05:09,440 --> 00:05:11,990 process and partnership with Chief Data Officers 138 00:05:11,990 --> 00:05:13,690 from across the IC, 139 00:05:13,690 --> 00:05:15,090 and addresses data management 140 00:05:15,090 --> 00:05:16,480 throughout the data life cycle, 141 00:05:16,480 --> 00:05:17,390 which Keith will try 142 00:05:17,390 --> 00:05:19,160 and show you on the board in a minute. 143 00:05:19,160 --> 00:05:21,300 We took the DAMA Data Life Cycle Wheel 144 00:05:21,300 --> 00:05:22,300 and tweaked it a little bit 145 00:05:22,300 --> 00:05:24,490 for Intelligence Community purposes, 146 00:05:24,490 --> 00:05:26,630 and I think you heard one of our NSA teams 147 00:05:26,630 --> 00:05:28,870 talk about this in one of their breakout sessions as well. 148 00:05:28,870 --> 00:05:30,280 Slightly different graphic, 149 00:05:30,280 --> 00:05:31,700 but same general gist there 150 00:05:31,700 --> 00:05:33,213 of what are the things we need to do 151 00:05:33,213 --> 00:05:36,910 that data permeates this entire life cycle? 152 00:05:36,910 --> 00:05:39,320 From an IC standpoint in the ODNI, 153 00:05:39,320 --> 00:05:41,550 data permeates everything we do. 154 00:05:41,550 --> 00:05:42,730 In 2018, 155 00:05:42,730 --> 00:05:44,040 we published our strategy 156 00:05:44,040 --> 00:05:46,220 for Augmenting Intelligence with Machines, 157 00:05:46,220 --> 00:05:47,830 or the AIM Strategy. 158 00:05:47,830 --> 00:05:49,220 I don't have copies with me, 159 00:05:49,220 --> 00:05:51,090 but if you go to https://odni.gov 160 00:05:51,090 --> 00:05:53,690 you can find the document right there. 161 00:05:53,690 --> 00:05:55,840 Four big ideas related to AIM. 162 00:05:55,840 --> 00:05:58,320 It's building the digital foundation, 163 00:05:58,320 --> 00:06:01,820 rapidly adopting commercial and open source AI solutions, 164 00:06:01,820 --> 00:06:05,240 be fast followers, invest in the gaps. 165 00:06:05,240 --> 00:06:07,300 So in the light of cyber security, 166 00:06:07,300 --> 00:06:09,660 what are we gonna do about AI assurance? 167 00:06:09,660 --> 00:06:10,800 How do we know it works? 168 00:06:10,800 --> 00:06:12,340 How do we know it's safe and secure? 169 00:06:12,340 --> 00:06:15,570 How do we know that model or that analytic is authoritative? 170 00:06:15,570 --> 00:06:18,600 Some tough questions to talk about as we look to the future. 171 00:06:18,600 --> 00:06:20,250 And it really articulates the vision 172 00:06:20,250 --> 00:06:21,970 for what the IC needs to do 173 00:06:21,970 --> 00:06:25,480 to respond to the global challenges of overwhelming data, 174 00:06:25,480 --> 00:06:27,570 rapid advances in technology, 175 00:06:27,570 --> 00:06:29,300 and dealing with aggressive adversaries 176 00:06:29,300 --> 00:06:30,600 like Russia and China, 177 00:06:30,600 --> 00:06:32,580 and I pretty much heard that in every keynote, 178 00:06:32,580 --> 00:06:33,890 in every session this week. 179 00:06:33,890 --> 00:06:37,560 I think we're in lock step with our DIA and DOD colleagues 180 00:06:37,560 --> 00:06:39,490 on these are hot topics we all 181 00:06:39,490 --> 00:06:41,290 have to address together. 182 00:06:41,290 --> 00:06:44,060 So there's some great activities underway in DOD's, 183 00:06:44,060 --> 00:06:45,110 I'm gonna pronounce it incorrectly, 184 00:06:45,110 --> 00:06:48,160 the JAIC, the Joint Artificial Intelligence Center, 185 00:06:48,160 --> 00:06:50,710 and IC's emerging AIM Hub, 186 00:06:50,710 --> 00:06:52,640 along with some innovation outposts 187 00:06:52,640 --> 00:06:54,600 as part of our IC partners 188 00:06:54,600 --> 00:06:57,410 on how do we collaborate and integrate faster? 189 00:06:57,410 --> 00:06:59,380 We need more intentional joint projects, 190 00:06:59,380 --> 00:07:01,510 and frankly we need more joint duty assignments. 191 00:07:01,510 --> 00:07:03,170 And I've seen some of that here. 192 00:07:03,170 --> 00:07:05,150 There's some great partnerships going on. 193 00:07:05,150 --> 00:07:06,860 Need to think about those opportunities 194 00:07:06,860 --> 00:07:09,713 so we can be more purple as go forward on this. 195 00:07:10,860 --> 00:07:13,410 Many of you have heard about the three V's of data, 196 00:07:13,410 --> 00:07:15,500 volume, variety, and velocity. 197 00:07:15,500 --> 00:07:18,040 For IC purposes, we expand that to two more, 198 00:07:18,040 --> 00:07:20,900 adding in validity and veracity. 199 00:07:20,900 --> 00:07:23,030 This means that the solutions we develop, 200 00:07:23,030 --> 00:07:25,230 and this means you, too, industry partners, 201 00:07:25,230 --> 00:07:27,130 academic and foreign liaison partners 202 00:07:27,130 --> 00:07:28,810 who are in the room today, 203 00:07:28,810 --> 00:07:31,010 need to address all five of those dimensions 204 00:07:31,010 --> 00:07:33,203 as you help us work on those solutions. 205 00:07:34,630 --> 00:07:35,490 Early this year, 206 00:07:35,490 --> 00:07:37,130 the IC also published the new 207 00:07:37,130 --> 00:07:39,180 National Intelligence Strategy, 208 00:07:39,180 --> 00:07:41,450 which outlines the key strategies for the IC 209 00:07:41,450 --> 00:07:43,650 about information sharing and safe guarding, 210 00:07:43,650 --> 00:07:45,550 also available on https://odni.gov. 211 00:07:45,550 --> 00:07:48,410 And that really drives the top-down strategy 212 00:07:48,410 --> 00:07:49,277 above what we're doing here 213 00:07:49,277 --> 00:07:51,520 and with our Eyesight partners. 214 00:07:51,520 --> 00:07:52,590 And earlier this week, 215 00:07:52,590 --> 00:07:55,490 you heard from our IC CIO, Chief Information Officer, 216 00:07:55,490 --> 00:07:57,710 John Sherman, who's right here in the front row, 217 00:07:57,710 --> 00:08:00,470 about the new strategic plan to advance Cloud Computing 218 00:08:00,470 --> 00:08:02,700 in the IC that was approved in June. 219 00:08:02,700 --> 00:08:04,217 You've been to a couple of breakout sessions on it, 220 00:08:04,217 --> 00:08:05,880 and we have copies in the back, 221 00:08:05,880 --> 00:08:08,210 and you can also scan the QR code for that document 222 00:08:08,210 --> 00:08:10,090 as well today. 223 00:08:10,090 --> 00:08:12,980 Together, these strategy documents lay out our vision. 224 00:08:12,980 --> 00:08:15,830 And if you have heard DNI Leadership recently, 225 00:08:15,830 --> 00:08:18,750 then you know that the landscape is rapidly changing. 226 00:08:18,750 --> 00:08:20,930 Today, as the IC and across DOD, 227 00:08:20,930 --> 00:08:24,520 we have to embrace an age of ubiquitous technology, 228 00:08:24,520 --> 00:08:27,500 digital connectiveness, and data abundance. 229 00:08:27,500 --> 00:08:28,880 We look forward to partnering with you 230 00:08:28,880 --> 00:08:30,080 on our journey together. 231 00:08:31,390 --> 00:08:32,800 - Thank you very much, Nancy, 232 00:08:32,800 --> 00:08:36,970 for those very inspiring and insightful comments. 233 00:08:36,970 --> 00:08:38,020 Michael, over to you. 234 00:08:39,207 --> 00:08:40,870 - (clears throat) Excuse me. 235 00:08:40,870 --> 00:08:42,560 Good morning, everyone. 236 00:08:42,560 --> 00:08:46,520 Couple things that Dave didn't mention. 237 00:08:46,520 --> 00:08:48,470 First, I'm a Commercial Sector Person. 238 00:08:48,470 --> 00:08:51,730 I've spent most of my career over in the Commercial Sector. 239 00:08:51,730 --> 00:08:55,060 I'm still very much a stranger in a strange land 240 00:08:55,060 --> 00:08:57,400 when it comes to getting things done productively 241 00:08:57,400 --> 00:09:00,300 in government as Pat and others can tell you. 242 00:09:00,300 --> 00:09:02,890 There's a lot of things about the way government 243 00:09:02,890 --> 00:09:04,783 and the Department work that, 244 00:09:05,890 --> 00:09:08,680 and no matter how long I am employed by the Department, 245 00:09:08,680 --> 00:09:10,970 I will still be pondering. 246 00:09:10,970 --> 00:09:14,010 So I'm well aware of the limits of my knowledge. 247 00:09:14,010 --> 00:09:15,440 I like to say that they hired me 248 00:09:15,440 --> 00:09:16,890 because I'm a Commercial Sector guy, 249 00:09:16,890 --> 00:09:19,470 so my super power is I never worked for government 250 00:09:19,470 --> 00:09:20,910 before this job. 251 00:09:20,910 --> 00:09:23,130 All of you know that that means my super weakness 252 00:09:23,130 --> 00:09:25,650 is I never worked for government before this job. 253 00:09:25,650 --> 00:09:27,943 And so I'm aware of that, too. 254 00:09:29,438 --> 00:09:32,670 We have many of the same challenges that you have. 255 00:09:32,670 --> 00:09:34,993 We have certainly the same adversaries. 256 00:09:34,993 --> 00:09:38,000 We're, have that very much in common. 257 00:09:38,000 --> 00:09:39,960 And then we have the additional ones that come 258 00:09:39,960 --> 00:09:42,650 from being half of the federal government's budget. 259 00:09:42,650 --> 00:09:44,120 We have the additional ones that come 260 00:09:44,120 --> 00:09:45,710 from having the highest degree 261 00:09:45,710 --> 00:09:49,190 of multi-dimensional complexity of anything I've ever seen 262 00:09:49,190 --> 00:09:50,453 anywhere on the planet. 263 00:09:51,380 --> 00:09:52,443 And yet, 264 00:09:53,391 --> 00:09:57,110 we typically can find that an excuse 265 00:09:57,110 --> 00:09:58,290 not to collaborate, 266 00:09:58,290 --> 00:09:59,280 not to communicate, 267 00:09:59,280 --> 00:10:01,260 not to work together with our colleagues 268 00:10:01,260 --> 00:10:03,870 in the national security space, 269 00:10:03,870 --> 00:10:07,440 and we all know that that's not the path to success. 270 00:10:07,440 --> 00:10:10,230 That in fact, we must work together very closely. 271 00:10:10,230 --> 00:10:12,500 And it's nice to be at a point where we've started 272 00:10:12,500 --> 00:10:14,420 to identify very clear initiatives 273 00:10:14,420 --> 00:10:17,940 and spaces where we will take collaborative action. 274 00:10:17,940 --> 00:10:19,930 Otherwise, in the DOD, 275 00:10:19,930 --> 00:10:24,100 we get really wrapped around our own problems, 276 00:10:24,100 --> 00:10:27,700 around the fact that we have over 3,000 data centers, 277 00:10:27,700 --> 00:10:31,520 that we have, the public number is roughly, 278 00:10:31,520 --> 00:10:32,870 you know, like that, right? 279 00:10:32,870 --> 00:10:37,870 Roughly 10,000 Operational Systems of Applications. 280 00:10:38,000 --> 00:10:41,490 We say "roughly" because it lets us round down 281 00:10:41,490 --> 00:10:43,493 from a significantly higher number. 282 00:10:44,560 --> 00:10:47,110 Looking at that as an IT professional, 283 00:10:47,110 --> 00:10:49,913 and I am a technologist here, 284 00:10:50,750 --> 00:10:51,873 it's a hairball of IT. 285 00:10:52,860 --> 00:10:55,430 It is an absolute hairball. 286 00:10:55,430 --> 00:10:58,200 And getting data to the people who need it, 287 00:10:58,200 --> 00:11:01,937 assuring the veracity of the data, 288 00:11:01,937 --> 00:11:03,900 and the validity of the data, 289 00:11:03,900 --> 00:11:07,910 is a non-trivial challenge in an organization 290 00:11:07,910 --> 00:11:11,080 that disperses authority to align it 291 00:11:11,080 --> 00:11:13,670 with responsibility everywhere. 292 00:11:13,670 --> 00:11:17,740 And so, I have those additional challenges, 293 00:11:17,740 --> 00:11:20,610 but I don't want any of that to ever get in the way 294 00:11:20,610 --> 00:11:23,010 of recognizing we have a common challenge 295 00:11:23,010 --> 00:11:25,240 and a common opportunity. 296 00:11:25,240 --> 00:11:26,780 I don't want any of that to get in the way 297 00:11:26,780 --> 00:11:29,300 of us collaborating effectively together. 298 00:11:29,300 --> 00:11:30,648 I'll pause there. 299 00:11:30,648 --> 00:11:32,120 - Thank you, Michael. 300 00:11:32,120 --> 00:11:35,940 And when I started as the CDO for DIA, 301 00:11:35,940 --> 00:11:36,910 what I found was, 302 00:11:36,910 --> 00:11:38,880 I found that the authorities 303 00:11:38,880 --> 00:11:41,887 and the clarity coming out of the IC to me 304 00:11:41,887 --> 00:11:45,460 as an IC member was very clear. 305 00:11:45,460 --> 00:11:48,270 What I found on the DOD side, it was less clear. 306 00:11:48,270 --> 00:11:50,950 So we certainly established an open collaboration 307 00:11:50,950 --> 00:11:52,570 with the DOD, 308 00:11:52,570 --> 00:11:54,160 and thanks to the partnership as well, 309 00:11:54,160 --> 00:11:55,850 with John Turner and USDI, 310 00:11:55,850 --> 00:11:59,820 we've started to shore up our understanding of authorities 311 00:11:59,820 --> 00:12:01,360 and the conversation around: 312 00:12:01,360 --> 00:12:05,325 how do authorities for CEO's that are in law now 313 00:12:05,325 --> 00:12:09,560 convey down through the Department down to the, 314 00:12:09,560 --> 00:12:13,120 down to the agencies such as DIA? 315 00:12:13,120 --> 00:12:16,340 So John Turner, opening comments please. 316 00:12:16,340 --> 00:12:17,880 - Sure. 317 00:12:17,880 --> 00:12:21,620 So I think Dave's comments about the authorities piece 318 00:12:21,620 --> 00:12:24,110 is indicative of what we see 319 00:12:24,110 --> 00:12:26,370 across the IC Space 320 00:12:26,370 --> 00:12:28,270 when it comes to data governance and data management, 321 00:12:28,270 --> 00:12:31,060 which is, the IC's really leading the way. 322 00:12:31,060 --> 00:12:36,060 Across the federal government, the 17 Intel Agencies, 323 00:12:36,170 --> 00:12:37,610 and the IC Chief Data Officer 324 00:12:37,610 --> 00:12:39,830 really have made tremendous strides. 325 00:12:39,830 --> 00:12:43,440 And so USDI's role is bridging 326 00:12:43,440 --> 00:12:46,520 between the Department and the IC. 327 00:12:46,520 --> 00:12:50,230 It's been a real pleasure to take a lot of the great work 328 00:12:50,230 --> 00:12:51,880 that's been done in the IC 329 00:12:51,880 --> 00:12:54,430 and provide that to the Department writ large 330 00:12:55,523 --> 00:12:57,060 so that we have a doctrine 331 00:12:57,060 --> 00:12:59,220 within the Defense Intelligence Components 332 00:13:01,825 --> 00:13:05,120 on the MIP side as well, but principally, 333 00:13:05,120 --> 00:13:07,950 so that we can have alignment across DOD writ large. 334 00:13:07,950 --> 00:13:08,803 And so, 335 00:13:10,450 --> 00:13:13,540 where the IC has started with mission, 336 00:13:13,540 --> 00:13:15,260 and looking at mission data first, 337 00:13:15,260 --> 00:13:17,480 and then acknowledging that the IC 338 00:13:17,480 --> 00:13:19,890 needs to also tackle business data, 339 00:13:19,890 --> 00:13:22,200 the Department has started off 340 00:13:22,200 --> 00:13:24,130 because of Congressional direction 341 00:13:24,130 --> 00:13:26,530 and a number of issues that started off 342 00:13:26,530 --> 00:13:29,120 on the business side first, 343 00:13:29,120 --> 00:13:30,710 and then going to mission, right? 344 00:13:30,710 --> 00:13:34,650 But because of the National Defense Strategy, 345 00:13:34,650 --> 00:13:37,540 and Line of Effort one being increasing lethality, 346 00:13:37,540 --> 00:13:40,900 this is, this is a must do, right? 347 00:13:40,900 --> 00:13:43,370 We have to look at how our data is getting 348 00:13:43,370 --> 00:13:46,393 into weapon systems and increasing lethality, 349 00:13:47,230 --> 00:13:49,057 and you know, that's a hard problem, 350 00:13:49,057 --> 00:13:50,413 that's a hard challenge. 351 00:13:51,320 --> 00:13:52,410 - Great, thank you, John. 352 00:13:52,410 --> 00:13:55,200 And Line of Effort one, increasing lethality, 353 00:13:55,200 --> 00:13:57,530 if we think about that as a focus for the first question 354 00:13:57,530 --> 00:13:59,760 for our panel members here today, 355 00:13:59,760 --> 00:14:02,433 we'll go ahead and get started with some questions. 356 00:14:03,750 --> 00:14:04,583 So team, 357 00:14:04,583 --> 00:14:07,720 there's much discussion about Data Asset Inventories 358 00:14:07,720 --> 00:14:08,623 across government. 359 00:14:09,620 --> 00:14:10,940 What is the value, 360 00:14:10,940 --> 00:14:14,830 and how will we smartly inventory our data at scale, 361 00:14:14,830 --> 00:14:17,000 given the growth and volume? 362 00:14:17,000 --> 00:14:18,773 And why don't we start with you, Michael? 363 00:14:22,910 --> 00:14:26,120 - So this is where I reinforce your view 364 00:14:26,120 --> 00:14:27,173 that I'm a heretic. 365 00:14:31,734 --> 00:14:34,750 And if I take your question in the context 366 00:14:34,750 --> 00:14:39,750 of the Department's roughly 10,000 Systems of Applications, 367 00:14:41,912 --> 00:14:45,700 inventorying that in a practical way, 368 00:14:45,700 --> 00:14:50,540 when those roughly 10,000 Operation Systems of Applications 369 00:14:50,540 --> 00:14:54,360 are distributed across 1,500 subnets, 370 00:14:54,360 --> 00:14:59,150 separated by 502,219 371 00:14:59,150 --> 00:15:01,510 switches, routers, and firewalls, 372 00:15:01,510 --> 00:15:04,430 becomes a nontrivial challenge, right? 373 00:15:04,430 --> 00:15:07,640 There's no way to get there from the outside, 374 00:15:07,640 --> 00:15:09,610 you can only do it from the inside. 375 00:15:09,610 --> 00:15:12,250 And the environment owners, the system owners, 376 00:15:12,250 --> 00:15:13,420 the data owners, 377 00:15:13,420 --> 00:15:15,210 all have to be willing to cooperate 378 00:15:15,210 --> 00:15:16,670 to permit you to do that. 379 00:15:16,670 --> 00:15:19,352 Doesn't matter what tools and techniques you wanna use, 380 00:15:19,352 --> 00:15:22,473 the first approach is always persuasion. 381 00:15:23,960 --> 00:15:26,240 But those numbers alone tell you 382 00:15:26,240 --> 00:15:28,150 that we have duplication. 383 00:15:28,150 --> 00:15:31,940 That we have excess capability, 384 00:15:31,940 --> 00:15:33,630 that we have replication, 385 00:15:33,630 --> 00:15:36,400 and that's true of the data, as well. 386 00:15:36,400 --> 00:15:39,230 Which then calls into question the usefulness of the data. 387 00:15:39,230 --> 00:15:42,830 So my assumption is that you're not gonna go get it all, 388 00:15:42,830 --> 00:15:43,967 we're not gonna go get it all, 389 00:15:43,967 --> 00:15:47,540 but we're gonna make very carefully selected choices 390 00:15:47,540 --> 00:15:49,930 on instead of drift-netting the Department's data, 391 00:15:49,930 --> 00:15:52,700 we're gonna go spear fishing for the Department's data. 392 00:15:52,700 --> 00:15:56,330 It starts from what are the most important weapons platforms 393 00:15:56,330 --> 00:15:59,160 that we need to understand effectively in the kill chain, 394 00:15:59,160 --> 00:16:01,400 from sensor to shooter, 395 00:16:01,400 --> 00:16:04,810 and what are the ones that we need to improve our, 396 00:16:04,810 --> 00:16:06,900 either our resiliency in the field, 397 00:16:06,900 --> 00:16:11,900 or our other measures of usefulness of a weapons platform, 398 00:16:13,040 --> 00:16:15,477 and then we're gonna go after that data. 399 00:16:15,477 --> 00:16:18,910 And we're going to improve the quality of that data. 400 00:16:18,910 --> 00:16:20,677 We're gonna improve the linkage of that data, 401 00:16:20,677 --> 00:16:22,770 and we're gonna work our way forward 402 00:16:22,770 --> 00:16:25,480 based on the mission priorities 403 00:16:25,480 --> 00:16:27,390 of the leaders of the Department. 404 00:16:27,390 --> 00:16:29,410 I never start a project unless I know 405 00:16:29,410 --> 00:16:31,090 who the decision makers are that already 406 00:16:31,090 --> 00:16:32,540 consumed the output, 407 00:16:32,540 --> 00:16:36,590 and what they're looking for by way of new data, 408 00:16:36,590 --> 00:16:37,620 new insights, 409 00:16:37,620 --> 00:16:42,620 because my job is to help them make better decisions faster. 410 00:16:42,711 --> 00:16:46,230 Not to presume I know what decisions they should make. 411 00:16:46,230 --> 00:16:50,590 So we go after it in a very carefully focused 412 00:16:50,590 --> 00:16:52,270 and selected way. 413 00:16:52,270 --> 00:16:54,370 Will we ever build it up to where we want? 414 00:16:54,370 --> 00:16:55,203 No. 415 00:16:55,203 --> 00:16:57,600 Are we about to get spanked by the IG 416 00:16:57,600 --> 00:17:01,070 about a significant number of our platforms? 417 00:17:01,070 --> 00:17:01,903 Yes. 418 00:17:01,903 --> 00:17:03,020 Will we take that spanking 419 00:17:03,020 --> 00:17:05,100 and still focus on the priorities? 420 00:17:05,100 --> 00:17:06,220 Yes, absolutely. 421 00:17:06,220 --> 00:17:08,350 We're gonna focus on the priorities 422 00:17:08,350 --> 00:17:10,500 of the leaders of the Department. 423 00:17:10,500 --> 00:17:12,670 But we will make progress, 424 00:17:12,670 --> 00:17:14,140 and we will improve our ability 425 00:17:14,140 --> 00:17:17,100 to understand the data landscape 426 00:17:17,100 --> 00:17:19,850 and to manage the data landscape effectively, 427 00:17:19,850 --> 00:17:23,010 as long as it contributes directly to, in the end, 428 00:17:23,010 --> 00:17:26,531 Line of Effort one, in the National Defense Strategy. 429 00:17:26,531 --> 00:17:27,550 Is that a helpful answer? 430 00:17:27,550 --> 00:17:31,240 - Very good, and better decisions faster, 431 00:17:31,240 --> 00:17:33,010 I think is a great theme related 432 00:17:33,010 --> 00:17:35,599 to how do we improve lethality, as well. 433 00:17:35,599 --> 00:17:36,453 Nancy? 434 00:17:37,540 --> 00:17:40,296 - So it really starts with a very simple thought. 435 00:17:40,296 --> 00:17:41,910 If we don't know what data we have, 436 00:17:41,910 --> 00:17:43,110 how can we organize it? 437 00:17:43,110 --> 00:17:44,280 How can we safeguard it? 438 00:17:44,280 --> 00:17:45,558 How can we share it? 439 00:17:45,558 --> 00:17:49,040 If we already have that particular data set in our holdings, 440 00:17:49,040 --> 00:17:52,330 should be spend precious IC and DOD resources, 441 00:17:52,330 --> 00:17:54,660 and potentially put our people in harm's way, 442 00:17:54,660 --> 00:17:57,140 to collect more of that data? 443 00:17:57,140 --> 00:17:57,973 Alternatively, 444 00:17:57,973 --> 00:18:00,750 if we're already paying one element to collect the data, 445 00:18:00,750 --> 00:18:02,703 then how can we let other parts of the organization know 446 00:18:02,703 --> 00:18:04,660 that we already have the data, 447 00:18:04,660 --> 00:18:06,380 how they can get access to it, 448 00:18:06,380 --> 00:18:09,650 what time frame it covers, and so on. 449 00:18:09,650 --> 00:18:10,630 But there's more to it. 450 00:18:10,630 --> 00:18:13,440 We also have to think about the down stream phases. 451 00:18:13,440 --> 00:18:15,330 Who can use the data and under what conditions? 452 00:18:15,330 --> 00:18:16,890 I think this gets to a little different angle 453 00:18:16,890 --> 00:18:18,520 with the authorities question. 454 00:18:18,520 --> 00:18:20,830 Are there privacy and civil liberties implications 455 00:18:20,830 --> 00:18:23,250 to how the data is being kept and used? 456 00:18:23,250 --> 00:18:25,700 And what are the policy implications when we fuse 457 00:18:25,700 --> 00:18:29,300 all this data collected under multiple authorities? 458 00:18:29,300 --> 00:18:31,530 Are we authorized to keep this data forever? 459 00:18:31,530 --> 00:18:33,210 And coming from a records management world, 460 00:18:33,210 --> 00:18:35,080 that means life of the Republic. 461 00:18:35,080 --> 00:18:36,250 How do we do that? 462 00:18:36,250 --> 00:18:38,740 How do we insure that content is still discoverable 463 00:18:38,740 --> 00:18:39,630 and retrievable? 464 00:18:39,630 --> 00:18:41,540 Industry partners, that includes you. 465 00:18:41,540 --> 00:18:43,340 Some of you have had some tough conversations 466 00:18:43,340 --> 00:18:44,373 with me about that. 467 00:18:45,400 --> 00:18:49,050 In terms of 75 years for a Records Retention Schedule, 468 00:18:49,050 --> 00:18:50,080 how do you really do that 469 00:18:50,080 --> 00:18:52,640 and ensure those encryption keys are still gonna work? 470 00:18:52,640 --> 00:18:54,107 I think one company comment to me, 471 00:18:54,107 --> 00:18:55,934 "Our developers aren't that old yet." 472 00:18:55,934 --> 00:18:56,767 (audience murmurs) 473 00:18:56,767 --> 00:18:57,600 That's not the right answer. 474 00:18:57,600 --> 00:19:00,410 That may be true, but that's not the right answer. 475 00:19:00,410 --> 00:19:02,223 So help us figure that out. 476 00:19:03,440 --> 00:19:05,170 And then for things like bulk data, 477 00:19:05,170 --> 00:19:07,190 we may have to follow things like Attorney General 478 00:19:07,190 --> 00:19:09,500 guidelines that say we have to age that off 479 00:19:09,500 --> 00:19:11,350 after a very specific period of time, 480 00:19:11,350 --> 00:19:13,450 and we can get sued when we don't do that. 481 00:19:14,470 --> 00:19:19,010 So the IC received a mandate from DNI Leadership in 2019 482 00:19:19,010 --> 00:19:23,040 to catalog all of its data sets by the end of 2019. 483 00:19:23,040 --> 00:19:25,210 So coming in new to the CDO Council, I said, 484 00:19:25,210 --> 00:19:27,580 was that end of fiscal year or calendar year? 485 00:19:27,580 --> 00:19:29,540 So I think we've bought ourselves three more months 486 00:19:29,540 --> 00:19:31,120 in the discussion. 487 00:19:31,120 --> 00:19:33,590 We've been actively pursuing this effort, 488 00:19:33,590 --> 00:19:36,510 but as Mike mentioned, it's proven to be quite challenging. 489 00:19:36,510 --> 00:19:38,440 We're learning a lot along the way, 490 00:19:38,440 --> 00:19:40,110 and one of the most important lessons is, 491 00:19:40,110 --> 00:19:41,370 you're never really done. 492 00:19:41,370 --> 00:19:43,950 A static one-time snapshot of your data 493 00:19:43,950 --> 00:19:47,050 really decays very quickly in terms of its value. 494 00:19:47,050 --> 00:19:48,530 So we're grappling with the questions 495 00:19:48,530 --> 00:19:51,750 of how does a stand alone catalog benefit? 496 00:19:51,750 --> 00:19:54,110 Is it a sufficient center of gravity 497 00:19:54,110 --> 00:19:55,330 for different data stewards, 498 00:19:55,330 --> 00:19:58,280 or whether and how a catalog needs to be an output 499 00:19:58,280 --> 00:20:01,120 of a more robust larger data governance system? 500 00:20:01,120 --> 00:20:02,640 You can see where the answer's heading. 501 00:20:02,640 --> 00:20:04,870 I've given you a clue, right there. 502 00:20:04,870 --> 00:20:07,210 Different stake holders need different data elements 503 00:20:07,210 --> 00:20:08,760 about a data set in the catalog, 504 00:20:08,760 --> 00:20:09,950 and we've had, I will say, 505 00:20:09,950 --> 00:20:12,630 robust discussions in the CDO crowd 506 00:20:12,630 --> 00:20:15,350 about who are the data catalog entries for? 507 00:20:15,350 --> 00:20:17,270 Are they for the data scientists? 508 00:20:17,270 --> 00:20:19,410 Are they for an all-source analyst? 509 00:20:19,410 --> 00:20:21,500 Are they for the people who handle ingestion 510 00:20:21,500 --> 00:20:25,320 for a major repository like our IC Data Services teams? 511 00:20:25,320 --> 00:20:27,380 Some of whom are in the audience today. 512 00:20:27,380 --> 00:20:29,740 What about compliance roles like civil liberties, 513 00:20:29,740 --> 00:20:32,440 privacy, legal, and records management? 514 00:20:32,440 --> 00:20:33,487 These aspects are different, 515 00:20:33,487 --> 00:20:35,330 and the collectors and data stewards 516 00:20:35,330 --> 00:20:38,430 want different things than some of those down stream roles, 517 00:20:38,430 --> 00:20:40,670 how do we achieve all that and make this the most 518 00:20:40,670 --> 00:20:43,640 valuable possible in terms of that entry? 519 00:20:43,640 --> 00:20:45,580 If you look at entries in our catalogs 520 00:20:45,580 --> 00:20:46,450 today in the community, 521 00:20:46,450 --> 00:20:48,620 some of them are fairly geeky, 522 00:20:48,620 --> 00:20:51,560 and the Data Conditioning ETL Adjustment teams 523 00:20:51,560 --> 00:20:52,960 really like those. 524 00:20:52,960 --> 00:20:55,680 Not sure the all-source analysts are finding that data 525 00:20:55,680 --> 00:20:57,000 valuable in a way, 526 00:20:57,000 --> 00:20:59,170 and we're also having robust discussions 527 00:20:59,170 --> 00:21:00,960 about what is a data set? 528 00:21:00,960 --> 00:21:03,060 Is that that single use, single person, 529 00:21:03,060 --> 00:21:05,720 spreadsheet that tracks particular set of ships 530 00:21:05,720 --> 00:21:07,560 going through a particular canal? 531 00:21:07,560 --> 00:21:08,960 Is that a data set? 532 00:21:08,960 --> 00:21:11,500 Is it a whole stream of, you know, 533 00:21:11,500 --> 00:21:13,540 I94 landing cards coming in 534 00:21:13,540 --> 00:21:15,640 and who's crossing the border? 535 00:21:15,640 --> 00:21:18,690 Where's the boundary in between, or all of them valid? 536 00:21:18,690 --> 00:21:21,180 We also have to find ways to apply automation, 537 00:21:21,180 --> 00:21:22,440 just like Mike said. 538 00:21:22,440 --> 00:21:24,850 We don't have enough humans to keep up with the volume 539 00:21:24,850 --> 00:21:26,620 of data we're already swimming in, 540 00:21:26,620 --> 00:21:28,750 the volume of data coming our way. 541 00:21:28,750 --> 00:21:30,610 If any of you've seen the stats each year 542 00:21:30,610 --> 00:21:33,360 for what gets produced in an internet minute, 543 00:21:33,360 --> 00:21:35,130 and I didn't bring my cheat sheet with me 544 00:21:35,130 --> 00:21:35,963 of just how much, 545 00:21:35,963 --> 00:21:37,220 and I've got a 19-year-old at home. 546 00:21:37,220 --> 00:21:39,930 He and his friends alone are producing a ridiculous 547 00:21:39,930 --> 00:21:42,180 amount of data in the internet minute, 548 00:21:42,180 --> 00:21:45,020 bringing our network at home to a crawl. 549 00:21:45,020 --> 00:21:47,120 But our industry partners are also helping us 550 00:21:47,120 --> 00:21:49,810 collect more data from more systems and sensors than ever. 551 00:21:49,810 --> 00:21:53,050 And that was abundantly clear to me at DOTUS this week 552 00:21:53,050 --> 00:21:55,800 of just how many cool programs we have underway, 553 00:21:55,800 --> 00:21:59,660 but streaming in more data in more formats than ever before. 554 00:21:59,660 --> 00:22:02,150 So how are we gonna deal with those challenges? 555 00:22:02,150 --> 00:22:03,650 And how can we do the systems? 556 00:22:03,650 --> 00:22:05,080 So how, 557 00:22:05,080 --> 00:22:07,700 when we first identify a particular data set, 558 00:22:07,700 --> 00:22:09,147 we have to sort through authorities 559 00:22:09,147 --> 00:22:10,890 and retentions questions. 560 00:22:10,890 --> 00:22:12,850 How can we capture those business rules 561 00:22:12,850 --> 00:22:14,580 right at the time up front? 562 00:22:14,580 --> 00:22:16,540 How can we have the systems mark that data 563 00:22:16,540 --> 00:22:19,880 in that way up front and then add more useful data? 564 00:22:19,880 --> 00:22:22,530 We were having sidebar discussions over the past two days 565 00:22:22,530 --> 00:22:24,570 about the meta-data elements that are of interest 566 00:22:24,570 --> 00:22:26,070 to one domain area, 567 00:22:26,070 --> 00:22:28,210 we may need some additional domain elements 568 00:22:28,210 --> 00:22:29,610 for another area, 569 00:22:29,610 --> 00:22:32,900 and how do we enrich that data throughout its life cycle? 570 00:22:32,900 --> 00:22:35,480 How do we seemly integrate this into our natural 571 00:22:35,480 --> 00:22:37,980 work processes for both new and existing? 572 00:22:37,980 --> 00:22:39,610 Brand new green field way forward, 573 00:22:39,610 --> 00:22:41,690 this is gonna be much easier to do. 574 00:22:41,690 --> 00:22:43,500 As a long time IT Program Manager, 575 00:22:43,500 --> 00:22:46,350 those Legacy Systems can be tough when we start to go 576 00:22:46,350 --> 00:22:48,820 under the covers and undo things. 577 00:22:48,820 --> 00:22:51,030 So really, part of my challenge to all of you is, 578 00:22:51,030 --> 00:22:53,340 how do we architect for change going forward? 579 00:22:53,340 --> 00:22:55,810 Bake that in up front knowing that we're going 580 00:22:55,810 --> 00:22:57,340 to have to enrich the data, 581 00:22:57,340 --> 00:22:59,680 knowing that we're gonna wanna do new and exciting things 582 00:22:59,680 --> 00:23:00,700 with that data, 583 00:23:00,700 --> 00:23:02,820 think differently about how we build our systems 584 00:23:02,820 --> 00:23:03,763 right up front. 585 00:23:05,040 --> 00:23:06,290 So over the next five years, 586 00:23:06,290 --> 00:23:08,620 we must transform how our data is discovered, 587 00:23:08,620 --> 00:23:10,050 accessed, and used, 588 00:23:10,050 --> 00:23:12,840 to make our informed decisions to protect our nation 589 00:23:12,840 --> 00:23:15,410 and respond to an ever changing landscape of threats. 590 00:23:15,410 --> 00:23:17,650 This cataloging effort is critical to that, 591 00:23:17,650 --> 00:23:19,190 but I'll take your challenge that, 592 00:23:19,190 --> 00:23:20,530 how will we ever know we're done? 593 00:23:20,530 --> 00:23:23,250 And at least one of our CDO's from my home agency, 594 00:23:23,250 --> 00:23:25,550 challenges me regularly on what's done 595 00:23:25,550 --> 00:23:27,230 when you're cataloging data? 596 00:23:27,230 --> 00:23:28,595 And I'll stop it there. 597 00:23:28,595 --> 00:23:30,550 - I know it's John turn, 598 00:23:30,550 --> 00:23:34,470 but I feel I need to foot stomp a couple of points you make, 599 00:23:34,470 --> 00:23:36,070 so important. - Please. 600 00:23:36,070 --> 00:23:38,220 - This idea of architecting for change, 601 00:23:38,220 --> 00:23:41,220 optimizing for change, absolutely critical. 602 00:23:41,220 --> 00:23:45,890 The mind set for many folks is that there is a destination. 603 00:23:45,890 --> 00:23:48,030 We reach that destination, we're done. 604 00:23:48,030 --> 00:23:50,570 And this is much more like physical fitness, 605 00:23:50,570 --> 00:23:52,990 where you've got to exercise on the routine basis, 606 00:23:52,990 --> 00:23:54,980 you've got to eat healthy on a routine basis, 607 00:23:54,980 --> 00:23:57,550 you've got to make choices about what beverage 608 00:23:57,550 --> 00:23:59,500 am I going to drink this evening with dinner 609 00:23:59,500 --> 00:24:00,923 on a routine basis. 610 00:24:02,290 --> 00:24:07,000 In this world, it's a constant change out of us. 611 00:24:07,000 --> 00:24:10,093 In the context of 75-year retention of data, 612 00:24:10,093 --> 00:24:12,600 will the encryption keys still be good, 613 00:24:12,600 --> 00:24:14,580 oh, hell no, they're not gonna be good in three years, 614 00:24:14,580 --> 00:24:17,130 because they'll be cracked in three years. 615 00:24:17,130 --> 00:24:19,250 But we haven't set up our systems 616 00:24:19,250 --> 00:24:22,440 to be constantly reviewing whether an encryption key 617 00:24:22,440 --> 00:24:26,560 is still secure enough for the data its applied to, 618 00:24:26,560 --> 00:24:29,440 and then to automatically roll it out 619 00:24:29,440 --> 00:24:32,460 and replace it with a different encryption key 620 00:24:32,460 --> 00:24:34,210 for that data so that we can keep it 621 00:24:34,210 --> 00:24:36,500 for another period of time. 622 00:24:36,500 --> 00:24:38,007 We assume that there's a safe space, 623 00:24:38,007 --> 00:24:40,080 and there are no safe spaces here 624 00:24:40,080 --> 00:24:41,840 because of the pace of change. 625 00:24:41,840 --> 00:24:44,870 And that is such an important mindset shift for us, 626 00:24:44,870 --> 00:24:47,140 because everything defaults to, 627 00:24:47,140 --> 00:24:49,470 oh, that's all I've got to do, 628 00:24:49,470 --> 00:24:50,780 once I've done it, I'm good. 629 00:24:50,780 --> 00:24:54,100 And the data catalog example's another great one. 630 00:24:54,100 --> 00:24:57,080 If that's not live feeds, if that's not federated, 631 00:24:57,080 --> 00:24:58,980 if it's not real time, 632 00:24:58,980 --> 00:25:01,350 it doesn't have any value whatsoever 633 00:25:01,350 --> 00:25:04,780 about three nanoseconds after we get it done. 634 00:25:04,780 --> 00:25:07,960 So thank you, those are such important points to make. 635 00:25:07,960 --> 00:25:08,793 - Thank you. 636 00:25:08,793 --> 00:25:10,266 - So and what I would add to that, 637 00:25:10,266 --> 00:25:12,420 thank you very much for those comments, 638 00:25:12,420 --> 00:25:14,460 what I would add to that is DIA is pursuing 639 00:25:14,460 --> 00:25:17,820 a data life cycle management as a service capability. 640 00:25:17,820 --> 00:25:19,840 I talked about this yesterday in our booth. 641 00:25:19,840 --> 00:25:23,610 And that means we have to integrate well with all systems 642 00:25:23,610 --> 00:25:25,360 that are data holders, 643 00:25:25,360 --> 00:25:28,530 and manage that data on their behalf 644 00:25:28,530 --> 00:25:30,227 over the course of its life cycle. 645 00:25:30,227 --> 00:25:32,190 And the point in time in which it's acquired 646 00:25:32,190 --> 00:25:33,930 through whatever means until a point in time 647 00:25:33,930 --> 00:25:35,230 in which it's retired. 648 00:25:35,230 --> 00:25:38,130 And it cannot be a standalone catalog that's off to the side 649 00:25:38,130 --> 00:25:40,790 with a bunch of manual entries from people. 650 00:25:40,790 --> 00:25:42,720 The other thing we talked about this morning at breakfast 651 00:25:42,720 --> 00:25:45,140 was metering the movement of data, 652 00:25:45,140 --> 00:25:47,220 understanding what's coming from where, 653 00:25:47,220 --> 00:25:48,850 what we're doing with it when we have it, 654 00:25:48,850 --> 00:25:50,900 and where we're sending it, you know? 655 00:25:50,900 --> 00:25:53,681 In my mind, you can't secure what you don't know you have. 656 00:25:53,681 --> 00:25:55,270 And think about a spill. 657 00:25:55,270 --> 00:25:57,550 Think about how you do you triage any damage 658 00:25:57,550 --> 00:26:00,260 associated with a spill if you don't have those answers? 659 00:26:00,260 --> 00:26:01,330 So really, in my mind, 660 00:26:01,330 --> 00:26:03,110 it's about data life cycle management, 661 00:26:03,110 --> 00:26:05,300 and it's about doing it as a service in a common, 662 00:26:05,300 --> 00:26:07,400 consistent manner for all applications. 663 00:26:07,400 --> 00:26:10,410 Building an integrator's guide so the systems are developed. 664 00:26:10,410 --> 00:26:12,810 You basically plug into that service 665 00:26:12,810 --> 00:26:16,010 so that those capabilities are managed in a consistent way 666 00:26:16,010 --> 00:26:17,594 for all applications. 667 00:26:17,594 --> 00:26:19,470 One other point I'll make, 668 00:26:19,470 --> 00:26:20,730 and then we'll move on to another question, 669 00:26:20,730 --> 00:26:21,760 and that is that, 670 00:26:21,760 --> 00:26:23,560 if you're an analyst, you know, 671 00:26:23,560 --> 00:26:26,000 one of the things that you render as part of your assessment 672 00:26:26,000 --> 00:26:27,610 is confidence. 673 00:26:27,610 --> 00:26:29,800 So the catalog, as it relates to lethality, 674 00:26:29,800 --> 00:26:31,200 is really about, well, 675 00:26:31,200 --> 00:26:34,070 your degree of confidence in your assessment, okay? 676 00:26:34,070 --> 00:26:36,580 Think about the data that you don't know about that 677 00:26:36,580 --> 00:26:39,170 could have affected your confidence in your assessment, 678 00:26:39,170 --> 00:26:43,180 either to the positive or to the negative, higher or lower. 679 00:26:43,180 --> 00:26:47,490 And we've seen this in the IC and in a number of cases, so, 680 00:26:47,490 --> 00:26:49,340 this is a very, very important area, 681 00:26:49,340 --> 00:26:52,430 knowing that what data's available to our analysts 682 00:26:52,430 --> 00:26:55,050 so they can render the highest possible confidence 683 00:26:55,050 --> 00:26:56,420 in their assessments, 684 00:26:56,420 --> 00:27:00,480 which of course leads to very pertinent, accurate, 685 00:27:00,480 --> 00:27:02,380 and timely intelligence. 686 00:27:02,380 --> 00:27:04,520 We'll move on to question two. - Dave. 687 00:27:04,520 --> 00:27:06,070 Before we go on to the next question, 688 00:27:06,070 --> 00:27:08,800 lemme just throw in two other thoughts here. 689 00:27:08,800 --> 00:27:13,800 The comments about live access to the data that we have, 690 00:27:14,000 --> 00:27:16,720 right, and not having it be a snapshot in time, are spot on. 691 00:27:16,720 --> 00:27:17,560 And, you know, 692 00:27:17,560 --> 00:27:20,320 I agree with Michael on we can't boil the ocean, 693 00:27:20,320 --> 00:27:21,600 and we're never gonna be done, 694 00:27:21,600 --> 00:27:24,150 and so we need to start with priorities, of course. 695 00:27:25,310 --> 00:27:28,780 I would add though, that what the IC, 696 00:27:28,780 --> 00:27:31,070 in my estimation, has demonstrated, 697 00:27:31,070 --> 00:27:35,210 is that there is tremendous value in doing the inventory 698 00:27:35,210 --> 00:27:38,110 even when it is a static snapshot in time, 699 00:27:38,110 --> 00:27:41,747 even when it is a broad and expansive scope, right? 700 00:27:41,747 --> 00:27:44,210 And that value proposition, I think, is two-fold. 701 00:27:44,210 --> 00:27:49,120 One, it's bringing enterprise maturity in terms 702 00:27:49,120 --> 00:27:52,230 of the work force across the board, 703 00:27:52,230 --> 00:27:53,590 understanding their roles 704 00:27:53,590 --> 00:27:55,840 and responsibilities for that data, right? 705 00:27:55,840 --> 00:27:59,210 Driving that conversation about who is a data custodian, 706 00:27:59,210 --> 00:28:00,290 who's a data steward, 707 00:28:00,290 --> 00:28:02,450 what are my responsibilities for this data? 708 00:28:02,450 --> 00:28:05,090 Those are foundational pieces that we need 709 00:28:05,090 --> 00:28:07,264 to work as an enterprise that are driven 710 00:28:07,264 --> 00:28:10,080 by the data inventory effort, right? 711 00:28:10,080 --> 00:28:11,730 So that has tremendous value. 712 00:28:11,730 --> 00:28:13,150 The other piece, too, 713 00:28:13,150 --> 00:28:16,900 is that as we're seeing across the board, 714 00:28:16,900 --> 00:28:20,098 there are innovative steps being taken 715 00:28:20,098 --> 00:28:21,980 where we're using data for things 716 00:28:21,980 --> 00:28:23,935 that were never anticipated before. 717 00:28:23,935 --> 00:28:28,010 So if we don't take a broad approach 718 00:28:28,010 --> 00:28:30,800 to cataloging as much of the data as we can, 719 00:28:30,800 --> 00:28:33,450 whether it's live or a snapshot in time, right, 720 00:28:33,450 --> 00:28:37,360 we're gonna miss opportunities for the innovation 721 00:28:37,360 --> 00:28:40,070 community and for other program managers 722 00:28:40,070 --> 00:28:43,380 to put things together and create new solutions 723 00:28:43,380 --> 00:28:45,140 to give us a symmetric advantage 724 00:28:46,140 --> 00:28:48,170 in ways that weren't anticipated. 725 00:28:48,170 --> 00:28:51,000 Right, so, we shouldn't lose that opportunity. 726 00:28:51,000 --> 00:28:52,310 - Good point. 727 00:28:52,310 --> 00:28:54,040 - Thank you very much. 728 00:28:54,040 --> 00:28:56,200 So another question for the panel. 729 00:28:56,200 --> 00:28:59,310 It seems organizations all see value in data. 730 00:28:59,310 --> 00:29:00,143 As such, 731 00:29:00,143 --> 00:29:01,720 everyone is building data solutions 732 00:29:01,720 --> 00:29:04,290 in data services architectures. 733 00:29:04,290 --> 00:29:07,410 Given the size of DOD and IC organizations, 734 00:29:07,410 --> 00:29:09,150 how are you balancing the development 735 00:29:09,150 --> 00:29:11,490 of data services between the needs to support 736 00:29:11,490 --> 00:29:13,700 mission specific capabilities, 737 00:29:13,700 --> 00:29:16,540 vice the need to consolidate and optimize 738 00:29:16,540 --> 00:29:18,550 in a fiscally constrained environment? 739 00:29:18,550 --> 00:29:20,100 And we'll start with you, John. 740 00:29:21,890 --> 00:29:22,833 - So, 741 00:29:24,760 --> 00:29:27,460 the IC's done a great job in driving 742 00:29:27,460 --> 00:29:30,830 data services from a centralized manner, right? 743 00:29:30,830 --> 00:29:32,680 Hasn't been a one-size-fits-all, 744 00:29:32,680 --> 00:29:37,357 but it has been a fairly top down, centrally managed, 745 00:29:37,357 --> 00:29:41,810 and driven approach from a resourcing, and requirements, 746 00:29:41,810 --> 00:29:44,470 and adoption perspective. 747 00:29:44,470 --> 00:29:46,963 So I think that that's been very positive. 748 00:29:47,820 --> 00:29:49,330 I don't anticipate us, 749 00:29:49,330 --> 00:29:51,770 or at least on the Department side writ-large, right, 750 00:29:51,770 --> 00:29:55,263 we haven't, we haven't made that same sort of move. 751 00:29:56,490 --> 00:30:00,400 But I would offer that as to vendors, 752 00:30:00,400 --> 00:30:03,530 and also to components, 753 00:30:03,530 --> 00:30:08,010 all too often we see our component specific challenges 754 00:30:08,010 --> 00:30:10,000 as unique, right? 755 00:30:10,000 --> 00:30:11,470 And as we move to the Cloud, 756 00:30:11,470 --> 00:30:14,710 and as we look at our building new systems, and data, 757 00:30:14,710 --> 00:30:15,830 and refactoring, 758 00:30:15,830 --> 00:30:19,330 and continuing on the digital transformation that we're on, 759 00:30:19,330 --> 00:30:21,000 we need to be very careful 760 00:30:21,000 --> 00:30:24,810 about building our own, 761 00:30:24,810 --> 00:30:27,140 and looking more towards where can we team, 762 00:30:27,140 --> 00:30:28,950 where do we have a common mission, 763 00:30:28,950 --> 00:30:32,470 where can we give up some of our autonomy 764 00:30:32,470 --> 00:30:35,490 in order to gain speed and integration? 765 00:30:35,490 --> 00:30:38,413 Right, so I think that at least on the Department side, 766 00:30:40,235 --> 00:30:41,560 and perhaps it's a general comment about, you know, 767 00:30:41,560 --> 00:30:46,490 all of federal government, I think we tend to go, 768 00:30:46,490 --> 00:30:50,000 our natural inclination is more component specific, 769 00:30:50,000 --> 00:30:53,080 and we should be trying to push wherever we can 770 00:30:53,080 --> 00:30:55,080 towards common adoption and utilization. 771 00:30:57,180 --> 00:30:58,013 - Michael. 772 00:30:59,591 --> 00:31:01,670 - I certainly agree with that, John, 773 00:31:01,670 --> 00:31:05,290 and I think we see very clear signs of that 774 00:31:05,290 --> 00:31:06,270 within the leaders. 775 00:31:06,270 --> 00:31:11,270 I've had a number of conversations with the Chiefs, 776 00:31:11,410 --> 00:31:13,950 and the other four-stars, 777 00:31:13,950 --> 00:31:18,340 and they frequently credit the prior Secretary Mattis 778 00:31:18,340 --> 00:31:20,810 and prior Deputy Secretary Shanahan, 779 00:31:20,810 --> 00:31:23,170 with opening their eyes to the commonality 780 00:31:23,170 --> 00:31:26,070 and the value of doing things in the common fashion, 781 00:31:26,070 --> 00:31:29,798 the value of adopting best practices out of other spaces, 782 00:31:29,798 --> 00:31:31,610 acting jointly, 783 00:31:31,610 --> 00:31:35,210 and being much less likely 784 00:31:35,210 --> 00:31:38,550 to wrap themselves in their Title 10 authorities 785 00:31:38,550 --> 00:31:43,374 and act as though things are separate and independent. 786 00:31:43,374 --> 00:31:46,160 And this comes out in not only conversations, 787 00:31:46,160 --> 00:31:48,380 but it comes out in the decision making 788 00:31:48,380 --> 00:31:50,560 and the directives that we see coming 789 00:31:50,560 --> 00:31:52,380 from the Department's leaders. 790 00:31:52,380 --> 00:31:56,070 Under Secretary Esper and Deputy Secretary Norquist, 791 00:31:56,070 --> 00:31:59,280 I think we're gonna see an even stronger move 792 00:31:59,280 --> 00:32:02,093 in this direction of acting in a common fashion. 793 00:32:03,497 --> 00:32:07,220 A shedding of this belief that we're different, 794 00:32:07,220 --> 00:32:09,150 we're unique, we're separate. 795 00:32:09,150 --> 00:32:10,120 We have 796 00:32:11,140 --> 00:32:13,660 over 300 797 00:32:13,660 --> 00:32:16,563 unique financial systems. 798 00:32:18,500 --> 00:32:20,930 We all work for the same employer. 799 00:32:20,930 --> 00:32:23,800 We all have the same set of treasury guidelines. 800 00:32:23,800 --> 00:32:26,720 We all have the same strictures. 801 00:32:26,720 --> 00:32:29,523 We don't need 300 of the dang things. 802 00:32:30,890 --> 00:32:33,690 We're moving, but as an example of progress, 803 00:32:33,690 --> 00:32:36,500 in the civilian human resource space, 804 00:32:36,500 --> 00:32:39,520 we're moving to a single instance of a software 805 00:32:39,520 --> 00:32:43,100 as a service subscription to one solution 806 00:32:43,100 --> 00:32:46,473 for all civilian HR across the Department of Defense. 807 00:32:49,270 --> 00:32:51,800 Completion date is in the next fiscal year. 808 00:32:51,800 --> 00:32:53,840 Early in the next fiscal year. 809 00:32:53,840 --> 00:32:56,560 Everything's in train, everything's in progress. 810 00:32:56,560 --> 00:32:59,807 And now the services are starting to say, 811 00:32:59,807 --> 00:33:02,901 "Well, I wonder if we can't move the military HR 812 00:33:02,901 --> 00:33:04,667 "into the same solution, 813 00:33:04,667 --> 00:33:07,720 "because that would give me the visibility that I need." 814 00:33:07,720 --> 00:33:10,900 So there's real progress being made in recognizing 815 00:33:10,900 --> 00:33:11,830 that in fact 816 00:33:12,720 --> 00:33:14,860 we go to war together, 817 00:33:14,860 --> 00:33:18,677 we are at peace together, we work together as a Department, 818 00:33:18,677 --> 00:33:21,100 and we need common solutions to common problems. 819 00:33:21,100 --> 00:33:23,170 We need a common view of our data, 820 00:33:23,170 --> 00:33:27,070 and so I'm very optimistic 821 00:33:27,070 --> 00:33:27,910 about the progress 822 00:33:27,910 --> 00:33:28,950 that we're making, 823 00:33:28,950 --> 00:33:32,000 and the way we are changing our views 824 00:33:32,000 --> 00:33:33,670 on not only the systems, 825 00:33:33,670 --> 00:33:35,513 but the data within those systems. 826 00:33:37,010 --> 00:33:37,843 - Thank you. 827 00:33:37,843 --> 00:33:38,710 Nancy? 828 00:33:38,710 --> 00:33:39,730 - Sure. 829 00:33:39,730 --> 00:33:40,563 As the IC, 830 00:33:40,563 --> 00:33:43,760 we use the IC CDO Council to help us identify 831 00:33:43,760 --> 00:33:47,250 and prioritize our investments for IC Data Services, 832 00:33:47,250 --> 00:33:48,610 and then feed that information 833 00:33:48,610 --> 00:33:51,320 to the larger ODNI Budget Process 834 00:33:51,320 --> 00:33:54,460 and Consolidated Intelligence Guidance Process. 835 00:33:54,460 --> 00:33:57,960 All 17 IC Elements have an active CDO in our Council, 836 00:33:57,960 --> 00:34:00,050 and we've had some quality time together this week 837 00:34:00,050 --> 00:34:01,540 among several of us. 838 00:34:01,540 --> 00:34:03,480 And each year we revisit the priorities 839 00:34:03,480 --> 00:34:06,240 as part of the Annual Budget Process. 840 00:34:06,240 --> 00:34:09,210 We look for innovation happening in one part of the IC 841 00:34:09,210 --> 00:34:11,150 and assess whether that is a capability 842 00:34:11,150 --> 00:34:12,660 that should be expanded, 843 00:34:12,660 --> 00:34:14,990 or turn it to what I would call an enterprise 844 00:34:14,990 --> 00:34:17,060 or flagship level capability. 845 00:34:17,060 --> 00:34:20,040 We partner with the IC CIO Council, and our counterparts, 846 00:34:20,040 --> 00:34:23,350 and other IC Elements to see if the capability or service 847 00:34:23,350 --> 00:34:25,590 already exists on a particular network. 848 00:34:25,590 --> 00:34:27,010 We're not alone, you're not alone in this, 849 00:34:27,010 --> 00:34:29,580 the IC has some of the same thing going on. 850 00:34:29,580 --> 00:34:31,890 We've built a lot of great systems and capabilities 851 00:34:31,890 --> 00:34:32,723 over the years, 852 00:34:32,723 --> 00:34:34,430 but they've grown up a little big organically, 853 00:34:34,430 --> 00:34:37,620 and now it's time to rationalize a little bit across that. 854 00:34:37,620 --> 00:34:39,800 So working under the auspices of Eyesight, 855 00:34:39,800 --> 00:34:42,890 and working in close partnership with the IC CIO, 856 00:34:42,890 --> 00:34:46,130 we identified a series of foundational data services 857 00:34:46,130 --> 00:34:47,660 that we need to put in place. 858 00:34:47,660 --> 00:34:49,280 There's some materials on the back table 859 00:34:49,280 --> 00:34:51,640 and another QR code so you can listen to them. 860 00:34:51,640 --> 00:34:53,660 Hopefully some of you were able to attend the breakout 861 00:34:53,660 --> 00:34:56,370 session with our NSA team earlier this week 862 00:34:56,370 --> 00:34:59,250 to learn more about some of the services here. 863 00:34:59,250 --> 00:35:00,570 But when we, in other cases, 864 00:35:00,570 --> 00:35:03,630 another IT service provider has identified a need, 865 00:35:03,630 --> 00:35:06,100 realized it was not their core competency, 866 00:35:06,100 --> 00:35:08,440 and asked the CDO and CIO Councils to see 867 00:35:08,440 --> 00:35:10,750 if this fit into their larger plan. 868 00:35:10,750 --> 00:35:11,583 Similarly, 869 00:35:11,583 --> 00:35:14,090 we've identified several critical dependencies 870 00:35:14,090 --> 00:35:17,100 on the identity, credentialing, and access management, 871 00:35:17,100 --> 00:35:19,150 or ICAM service provider team, 872 00:35:19,150 --> 00:35:21,402 who I have to say has been a fantastic partner 873 00:35:21,402 --> 00:35:23,910 for us on the data services side. 874 00:35:23,910 --> 00:35:26,220 Since authoritative attributes about the people 875 00:35:26,220 --> 00:35:29,400 are essential to our success for IC data services, 876 00:35:29,400 --> 00:35:32,800 and it may be a tag line, "Tag the people, tag the data." 877 00:35:32,800 --> 00:35:35,140 But that really is foundational to what we need 878 00:35:35,140 --> 00:35:36,530 to do in this space, 879 00:35:36,530 --> 00:35:37,920 and so they're a critical partner. 880 00:35:37,920 --> 00:35:39,940 And rather than having the data services teams 881 00:35:39,940 --> 00:35:40,960 have to build this, 882 00:35:40,960 --> 00:35:43,670 we can partner with those teams to do it. 883 00:35:43,670 --> 00:35:45,370 CIA and NSA were named 884 00:35:45,370 --> 00:35:48,110 as the Formal Service of Common Concern service providers 885 00:35:48,110 --> 00:35:51,480 of the IC data services back in 2017, 886 00:35:51,480 --> 00:35:54,660 and they are aggressively building out services on JWICS, 887 00:35:54,660 --> 00:35:57,750 and now working through CDO Council to prioritize 888 00:35:57,750 --> 00:36:00,040 what services they still need to build on JWICS, 889 00:36:00,040 --> 00:36:03,070 and what's next on the Secret and Unclassified Fabrics. 890 00:36:03,070 --> 00:36:05,100 That's where some of you can help us from the different 891 00:36:05,100 --> 00:36:06,590 departments and agencies. 892 00:36:06,590 --> 00:36:08,308 Help us understand what you need next, 893 00:36:08,308 --> 00:36:10,060 and then the service provider teams 894 00:36:10,060 --> 00:36:12,030 can help us out with the sequencing. 895 00:36:12,030 --> 00:36:14,420 In some cases, if I think about extract, transform, 896 00:36:14,420 --> 00:36:16,467 and load, that's great that I wanna load capability, 897 00:36:16,467 --> 00:36:19,130 but if I haven't done the extract and transform first, 898 00:36:19,130 --> 00:36:21,408 maybe there's not the point in building the load 899 00:36:21,408 --> 00:36:23,023 component up first. 900 00:36:24,370 --> 00:36:26,000 As part of Eyesight Epic Three, 901 00:36:26,000 --> 00:36:28,317 we need to deliver these multi-fabric data services 902 00:36:28,317 --> 00:36:31,740 and solutions to ensure that as our data is discoverable, 903 00:36:31,740 --> 00:36:34,320 accessible, and usable across the enterprise. 904 00:36:34,320 --> 00:36:36,450 And what I'm excited about is building these services 905 00:36:36,450 --> 00:36:38,340 on JWICS and the Secret Fabric. 906 00:36:38,340 --> 00:36:41,510 Also means our DOD counterparts get to take advantage 907 00:36:41,510 --> 00:36:43,490 of these services as well if they have access 908 00:36:43,490 --> 00:36:45,170 to those networks. 909 00:36:45,170 --> 00:36:47,420 We all have to do this at the speed of mission 910 00:36:47,420 --> 00:36:49,700 to enable that operational and decision advantage. 911 00:36:49,700 --> 00:36:51,370 I think that's been a constant theme I've heard 912 00:36:51,370 --> 00:36:52,850 throughout the week. 913 00:36:52,850 --> 00:36:55,180 But we do live in a resource-constrained environment, 914 00:36:55,180 --> 00:36:56,950 so I'm gonna foot stomp this again. 915 00:36:56,950 --> 00:36:59,180 We have to think through the mission-specific needs 916 00:36:59,180 --> 00:37:00,750 versus the corporate needs, 917 00:37:00,750 --> 00:37:03,910 and we have to figure out how do we get off aging, 918 00:37:03,910 --> 00:37:06,540 dare I say ancient, Legacy systems? 919 00:37:06,540 --> 00:37:08,510 Now my teams have heard me say this. 920 00:37:08,510 --> 00:37:11,010 Systems shouldn't be old enough to go to college. 921 00:37:11,010 --> 00:37:12,850 It's not a healthy thing. 922 00:37:12,850 --> 00:37:15,720 In this day and age, we need to be refreshing, 923 00:37:15,720 --> 00:37:17,440 taking advantage, rearchitecting, 924 00:37:17,440 --> 00:37:20,080 and optimizing, and architecting for change. 925 00:37:20,080 --> 00:37:22,570 Sadly, I just read a report from ONB that said 926 00:37:22,570 --> 00:37:25,070 there are federal systems that are getting ready 927 00:37:25,070 --> 00:37:25,903 for retirement, 928 00:37:25,903 --> 00:37:27,380 and I don't mean system retirement, 929 00:37:27,380 --> 00:37:29,960 I mean they're 50 plus years old. 930 00:37:29,960 --> 00:37:32,100 I don't know if we're as guilty of that over here, 931 00:37:32,100 --> 00:37:33,950 you might have a few on the hit list. 932 00:37:33,950 --> 00:37:36,881 That really terrifies me in terms of can those systems, 933 00:37:36,881 --> 00:37:39,010 they're so brittle if they're that old, 934 00:37:39,010 --> 00:37:40,540 to do the things, the modern things, 935 00:37:40,540 --> 00:37:42,090 we need to do with data? 936 00:37:42,090 --> 00:37:45,970 So I challenge all of you and our vendor partners to really, 937 00:37:45,970 --> 00:37:48,420 we have to think about getting off some of these 938 00:37:48,420 --> 00:37:50,916 Legacy systems so we free up the resource to do 939 00:37:50,916 --> 00:37:53,109 the new innovative things we need to do 940 00:37:53,109 --> 00:37:56,113 to meet those asymmetric threats and challenges. 941 00:37:57,210 --> 00:37:58,480 All right, we've already talked a little bit 942 00:37:58,480 --> 00:37:59,400 about architect for change, 943 00:37:59,400 --> 00:38:01,340 so I'm not gonna foot stomp that now. 944 00:38:01,340 --> 00:38:03,960 The other thing I would say for the IT Development teams 945 00:38:03,960 --> 00:38:04,920 in the room, 946 00:38:04,920 --> 00:38:07,430 you've gotta get there with agile development methodologies 947 00:38:07,430 --> 00:38:08,890 and really with the SecDEVOPS, 948 00:38:08,890 --> 00:38:09,950 or the Secure DEVOPS. 949 00:38:09,950 --> 00:38:12,880 That's something that's been a flagship of my success 950 00:38:12,880 --> 00:38:13,870 is the teams doing it, 951 00:38:13,870 --> 00:38:15,290 automating these deployments, 952 00:38:15,290 --> 00:38:17,610 baking in the security of those things, 953 00:38:17,610 --> 00:38:19,900 not having to spend your precious development dollars 954 00:38:19,900 --> 00:38:22,470 doing the sort of core software development things, 955 00:38:22,470 --> 00:38:23,930 but doing the hard mission things. 956 00:38:23,930 --> 00:38:26,370 That's what we need to bake into this. 957 00:38:26,370 --> 00:38:29,100 So as part of the greater IC Federated Approach, 958 00:38:29,100 --> 00:38:30,720 I hope some of you were able to attend 959 00:38:30,720 --> 00:38:32,820 Cynthia Mendoza's session yesterday 960 00:38:32,820 --> 00:38:34,800 on the data reference architecture, 961 00:38:34,800 --> 00:38:36,620 where we're starting to come up with a minimum set 962 00:38:36,620 --> 00:38:38,600 of standards and design patterns. 963 00:38:38,600 --> 00:38:39,860 Now we're a little bit further behind 964 00:38:39,860 --> 00:38:41,700 on the data reference architecture 965 00:38:41,700 --> 00:38:44,130 compared to the collaboration and ICAM architectures, 966 00:38:44,130 --> 00:38:46,830 but I'm excited about the work we have getting underway, 967 00:38:46,830 --> 00:38:48,340 over the next, not really getting underway, 968 00:38:48,340 --> 00:38:51,030 but really digging in to drive out similar level 969 00:38:51,030 --> 00:38:51,863 of architecture. 970 00:38:51,863 --> 00:38:54,300 So I think you called them the Skittle's Charts yesterday? 971 00:38:54,300 --> 00:38:56,570 So we can, too, have our Skittle's Charts. 972 00:38:56,570 --> 00:38:57,640 Although, those of you who know me, 973 00:38:57,640 --> 00:38:59,100 knows it's really all about the chocolate, 974 00:38:59,100 --> 00:39:01,700 so we're gonna have to talk about something else for these, 975 00:39:01,700 --> 00:39:04,460 maybe rather than the Skittle's Charts for ours. 976 00:39:04,460 --> 00:39:07,010 We need to foster the adoption of the enterprise tools 977 00:39:07,010 --> 00:39:07,843 and services, 978 00:39:07,843 --> 00:39:11,170 I think this is another case of "build once, reuse many." 979 00:39:11,170 --> 00:39:14,130 And bring that innovation that's happening on the edge 980 00:39:14,130 --> 00:39:15,890 back into these Enterprise Class teams 981 00:39:15,890 --> 00:39:18,830 so they can make it available in a scalable way 982 00:39:18,830 --> 00:39:20,320 across the enterprise, 983 00:39:20,320 --> 00:39:23,760 optimized to work in our commercial Cloud spaces, 984 00:39:23,760 --> 00:39:27,120 and in our high performance data and analytic environment 985 00:39:27,120 --> 00:39:29,390 on the GovCloud's side as well. 986 00:39:29,390 --> 00:39:31,130 So we're following the tenets outlined 987 00:39:31,130 --> 00:39:33,490 in the IC Data Strategy, 988 00:39:33,490 --> 00:39:35,460 and now we have to also tackle things 989 00:39:35,460 --> 00:39:37,060 like the Data Sharing MOU's. 990 00:39:37,060 --> 00:39:38,010 I gotta tell you, 991 00:39:38,010 --> 00:39:39,660 I'm not worried about the tactical teams 992 00:39:39,660 --> 00:39:41,340 solving the tactical problems. 993 00:39:41,340 --> 00:39:43,120 With the kind of talent I've seen here at DOTUS, 994 00:39:43,120 --> 00:39:44,310 we can do that. 995 00:39:44,310 --> 00:39:45,210 But I've gotta tell you, 996 00:39:45,210 --> 00:39:48,530 these MOU's and Data Sharing Agreements, in some cases, 997 00:39:48,530 --> 00:39:49,940 Eric Downs and I were talking about it, 998 00:39:49,940 --> 00:39:51,150 it's been three years plus, 999 00:39:51,150 --> 00:39:52,850 and I've personally been involved 1000 00:39:52,850 --> 00:39:55,240 in some of these three-year negotiation cycles. 1001 00:39:55,240 --> 00:39:57,500 We are keeping teams of lawyers very busy 1002 00:39:57,500 --> 00:39:58,960 at multiple agencies, 1003 00:39:58,960 --> 00:40:01,220 and we need to do that to work through the authorities, 1004 00:40:01,220 --> 00:40:03,310 the privacy, and civil liberties issues, 1005 00:40:03,310 --> 00:40:05,960 but we've gotta find ways to automate this war. 1006 00:40:05,960 --> 00:40:08,550 So how can we get some of that meta-data and catologing 1007 00:40:08,550 --> 00:40:11,900 information up front baked in at the point of collection, 1008 00:40:11,900 --> 00:40:14,060 at approval to collect, to make, 1009 00:40:14,060 --> 00:40:16,610 accelerate our data flowing through that data life cycle 1010 00:40:16,610 --> 00:40:18,620 to meet the mission needs. 1011 00:40:18,620 --> 00:40:20,430 And with that, one more thing. 1012 00:40:20,430 --> 00:40:22,150 If we can do this right, 1013 00:40:22,150 --> 00:40:25,370 we can also change how we do security control inheritance, 1014 00:40:25,370 --> 00:40:27,420 work towards A and A faster. 1015 00:40:27,420 --> 00:40:29,040 I still dream of ATO in a day. 1016 00:40:29,040 --> 00:40:30,780 I know we've talked about that in the IC, 1017 00:40:30,780 --> 00:40:32,680 but I don't think it's quite real yet. 1018 00:40:32,680 --> 00:40:34,600 I don't know if Sue or one of her folks is in the room, 1019 00:40:34,600 --> 00:40:35,860 Sue Door's team. 1020 00:40:35,860 --> 00:40:38,180 But we've gotta accelerate that cycle as well, 1021 00:40:38,180 --> 00:40:40,430 and thinking differently about how we build 1022 00:40:40,430 --> 00:40:42,080 the data services up front. 1023 00:40:42,080 --> 00:40:44,930 Using those data services on top of both of these Clouds 1024 00:40:44,930 --> 00:40:48,193 is going to accelerate your delivery into mission space. 1025 00:40:49,710 --> 00:40:50,860 - Very good. 1026 00:40:50,860 --> 00:40:51,693 Thank you, Nancy. 1027 00:40:51,693 --> 00:40:53,760 So what I'm gonna do, is I'm gonna take a, 1028 00:40:53,760 --> 00:40:55,490 a pause on questions from me, 1029 00:40:55,490 --> 00:40:57,630 and ask the audience if you have a few questions 1030 00:40:57,630 --> 00:40:59,480 you'd like to ask our esteemed panel. 1031 00:41:00,510 --> 00:41:01,860 If not, I have more, 1032 00:41:01,860 --> 00:41:03,677 but I wanna give you that opportunity. 1033 00:41:03,677 --> 00:41:05,022 (panel clamoring) 1034 00:41:05,022 --> 00:41:06,260 So in the very back? 1035 00:41:06,260 --> 00:41:08,084 - [Woman] Hi, good morning. 1036 00:41:08,084 --> 00:41:11,417 (woman mumbles off mic) 1037 00:41:13,577 --> 00:41:18,464 Any of you discussed enterprise solutions common, 1038 00:41:18,464 --> 00:41:21,390 can you expand on the desire to work towards 1039 00:41:21,390 --> 00:41:24,140 one CIA services or NSA services. 1040 00:41:25,180 --> 00:41:26,539 (woman mumbles off mic) 1041 00:41:26,539 --> 00:41:28,960 And often my request that I receive permission 1042 00:41:28,960 --> 00:41:31,806 from data sets that are owned by other (mumbles) teams, 1043 00:41:31,806 --> 00:41:35,410 and to try and get them to our (mumbles) 1044 00:41:35,410 --> 00:41:37,180 rather lengthy process. 1045 00:41:37,180 --> 00:41:38,510 So can you talk a little bit more 1046 00:41:38,510 --> 00:41:42,310 about trying to fill one solution that plays 1047 00:41:42,310 --> 00:41:44,150 well with the other solutions, 1048 00:41:44,150 --> 00:41:47,370 and the other catalogs that are prevalent in the IC? 1049 00:41:47,370 --> 00:41:48,770 - That's a multi-part question 1050 00:41:48,770 --> 00:41:51,050 that I think we have to unpack, so, 1051 00:41:51,050 --> 00:41:53,110 lemme take a crack at parts of it. 1052 00:41:53,110 --> 00:41:55,550 So one of the things we talked about is, I know we, 1053 00:41:55,550 --> 00:41:58,190 it sounds like we're building lots of different services, 1054 00:41:58,190 --> 00:41:59,920 and we are building multiple services 1055 00:41:59,920 --> 00:42:02,620 to handle different parts of that data life cycle there, 1056 00:42:02,620 --> 00:42:05,400 but we do believe we're building complimentary services. 1057 00:42:05,400 --> 00:42:07,890 But right now, we do have two different Clouds. 1058 00:42:07,890 --> 00:42:10,430 If you heard John's presentation earlier this week, 1059 00:42:10,430 --> 00:42:13,200 we have a high performance analytic compute environment 1060 00:42:13,200 --> 00:42:16,180 and we also have things running on the Amazon based 1061 00:42:16,180 --> 00:42:18,040 Cloud services in the IC Cloud. 1062 00:42:18,040 --> 00:42:18,980 So in some cases, 1063 00:42:18,980 --> 00:42:22,190 those services are operating to get data into one place 1064 00:42:22,190 --> 00:42:23,430 or the other to start. 1065 00:42:23,430 --> 00:42:25,580 But what the teams are aggressively working on is 1066 00:42:25,580 --> 00:42:27,730 how the data that comes in one place 1067 00:42:27,730 --> 00:42:29,530 make it work across the Clouds. 1068 00:42:29,530 --> 00:42:32,240 Develop analytics that work across the Clouds, and so on. 1069 00:42:32,240 --> 00:42:34,520 We're not quite there yet, I'll be candid, 1070 00:42:34,520 --> 00:42:35,930 but we are working aggressively 1071 00:42:35,930 --> 00:42:37,860 toward getting there on that. 1072 00:42:37,860 --> 00:42:39,960 I think that was one part of the question. 1073 00:42:42,320 --> 00:42:43,307 Help me out with the other parts of the question, 1074 00:42:43,307 --> 00:42:45,830 it was a little bit hard to hear down here. 1075 00:42:45,830 --> 00:42:46,830 - [Woman] Moving data sets. 1076 00:42:46,830 --> 00:42:48,010 - Yeah, so moving data sets. 1077 00:42:48,010 --> 00:42:50,520 So that is where the two data services teams 1078 00:42:50,520 --> 00:42:53,380 both have incredible out reach teams that can help you 1079 00:42:53,380 --> 00:42:55,950 with this and get started to help you with what do you 1080 00:42:55,950 --> 00:42:56,783 need to do, 1081 00:42:56,783 --> 00:42:58,870 and I think Vicky talked about it earlier this week, 1082 00:42:58,870 --> 00:43:02,150 or Scott in the team, they have sort of tiers of help, 1083 00:43:02,150 --> 00:43:03,310 in some cases we point you 1084 00:43:03,310 --> 00:43:05,400 to Reference Implementations and Documentations, 1085 00:43:05,400 --> 00:43:07,410 and I see Eric Downs nodding his head here. 1086 00:43:07,410 --> 00:43:09,750 But he's been through part of this on both sides. 1087 00:43:09,750 --> 00:43:10,583 In some cases, 1088 00:43:10,583 --> 00:43:12,170 teams can do this with a very light lift, 1089 00:43:12,170 --> 00:43:13,470 their technical team can handle 1090 00:43:13,470 --> 00:43:15,650 the Reference Implementation and get going, 1091 00:43:15,650 --> 00:43:17,240 and other teams need what I would call 1092 00:43:17,240 --> 00:43:18,347 more "Full Service", 1093 00:43:18,347 --> 00:43:19,930 help us bring the data in, 1094 00:43:19,930 --> 00:43:21,373 help us figure out how to condition it, 1095 00:43:21,373 --> 00:43:22,760 help us to get it in. 1096 00:43:22,760 --> 00:43:25,400 And so that range of services is available 1097 00:43:25,400 --> 00:43:26,560 to you through both teams, 1098 00:43:26,560 --> 00:43:29,110 and the agencies that have reached out to them 1099 00:43:29,110 --> 00:43:31,660 have had very strong and positive engagements with them. 1100 00:43:31,660 --> 00:43:34,340 So I really encourage you to reach out to the teams. 1101 00:43:34,340 --> 00:43:35,750 If you don't know how to contact them, 1102 00:43:35,750 --> 00:43:37,060 reach back to Dave, and I, and others, 1103 00:43:37,060 --> 00:43:39,613 and we will get you that contact information. 1104 00:43:40,670 --> 00:43:42,370 - Other comments on that question? 1105 00:43:44,740 --> 00:43:46,790 Okay, I think there was another question. 1106 00:43:48,170 --> 00:43:49,760 In the middle. 1107 00:43:49,760 --> 00:43:50,910 - [Man] Hi, my name is Scott Staff, 1108 00:43:50,910 --> 00:43:54,499 and I just want to mention the 3,000 data centers, 1109 00:43:54,499 --> 00:43:56,580 10,000 plus apps. 1110 00:43:56,580 --> 00:43:58,577 What other metrics and measures have you done 1111 00:43:58,577 --> 00:44:00,720 using the (mumbles). 1112 00:44:00,720 --> 00:44:02,560 - I'm sorry, I missed the last couple words. 1113 00:44:02,560 --> 00:44:03,610 What other metrics? 1114 00:44:03,610 --> 00:44:05,310 - What type of metrics and measures are you using 1115 00:44:05,310 --> 00:44:06,760 to base your decision making? 1116 00:44:08,040 --> 00:44:09,610 - To base decision making. 1117 00:44:09,610 --> 00:44:14,180 So decision making typically runs domain-by-domain 1118 00:44:14,180 --> 00:44:15,550 within a mission space, right? 1119 00:44:15,550 --> 00:44:16,720 So we have the war fighting mission, 1120 00:44:16,720 --> 00:44:17,830 we have the intelligence mission, 1121 00:44:17,830 --> 00:44:19,390 we have the business mission, 1122 00:44:19,390 --> 00:44:22,330 and then within those there are multiple domains. 1123 00:44:22,330 --> 00:44:23,970 In the business mission, for instance, 1124 00:44:23,970 --> 00:44:26,160 there's real property, there's medical, 1125 00:44:26,160 --> 00:44:28,670 there's acquisition, there's logistics and supply chain, 1126 00:44:28,670 --> 00:44:31,490 there's HR, there's finance and accounting, 1127 00:44:31,490 --> 00:44:34,253 there's information technology, there's cyber. 1128 00:44:35,370 --> 00:44:37,670 In every one of those spaces, 1129 00:44:37,670 --> 00:44:41,410 we are looking at different data 1130 00:44:41,410 --> 00:44:43,240 lined up against different decisions. 1131 00:44:43,240 --> 00:44:45,690 But it starts from what decisions are you trying 1132 00:44:45,690 --> 00:44:47,310 to make in the first place. 1133 00:44:47,310 --> 00:44:48,290 Who's that leader, 1134 00:44:48,290 --> 00:44:51,031 what decisions are they trying to make to improve 1135 00:44:51,031 --> 00:44:54,500 the performance or affordability of the organization? 1136 00:44:54,500 --> 00:44:57,447 That's a key phrase out of the National Defense Strategy. 1137 00:44:57,447 --> 00:45:00,350 Once you know those decisions they're trying to make, 1138 00:45:00,350 --> 00:45:01,720 then you have to ask them, 1139 00:45:01,720 --> 00:45:04,990 well, sir, what data do you need to make those decisions? 1140 00:45:04,990 --> 00:45:07,237 What are the critical questions you need answers to? 1141 00:45:07,237 --> 00:45:09,343 And we capture all those questions. 1142 00:45:10,260 --> 00:45:14,220 And then we start to go out with an aggressive 1143 00:45:14,220 --> 00:45:15,690 set of spear fishing, 1144 00:45:15,690 --> 00:45:18,470 looking for where that data sits. 1145 00:45:18,470 --> 00:45:21,140 Typically it sits in multiple places, 1146 00:45:21,140 --> 00:45:24,460 anywhere from 25 to 125 different places 1147 00:45:24,460 --> 00:45:25,800 we need to pull the data from. 1148 00:45:25,800 --> 00:45:27,830 Which, I will tell you, 1149 00:45:27,830 --> 00:45:31,550 we are relatively immature when it comes to the quality 1150 00:45:31,550 --> 00:45:32,600 of our data. 1151 00:45:32,600 --> 00:45:37,230 And my measure of that is, in those 25 to 125 spaces, 1152 00:45:37,230 --> 00:45:41,880 how many times do I find the same data schema being used 1153 00:45:41,880 --> 00:45:42,973 more than once? 1154 00:45:44,010 --> 00:45:47,223 The mathematical answer to that question is zero times. 1155 00:45:48,800 --> 00:45:50,810 That's how immature we are. 1156 00:45:50,810 --> 00:45:52,910 That means then that I need to be able to map 1157 00:45:52,910 --> 00:45:56,190 all of those different data schemas into a common one 1158 00:45:56,190 --> 00:46:00,150 that is specifically set up to answer the questions 1159 00:46:00,150 --> 00:46:03,560 to enable that decision maker to be effective. 1160 00:46:03,560 --> 00:46:05,870 And so we've developed a set of those. 1161 00:46:05,870 --> 00:46:10,260 We call them the CODE, or Common Decision Schemas. 1162 00:46:10,260 --> 00:46:13,620 But we've also aligned them against the commercial sector 1163 00:46:13,620 --> 00:46:15,430 bench marks in that space, 1164 00:46:15,430 --> 00:46:16,263 because of course, 1165 00:46:16,263 --> 00:46:19,450 the President's management agenda calls for government 1166 00:46:19,450 --> 00:46:21,770 to be as productive and as efficient in performing 1167 00:46:21,770 --> 00:46:24,060 business functions as the commercial sector 1168 00:46:24,060 --> 00:46:25,290 is productive and efficient. 1169 00:46:25,290 --> 00:46:28,178 So we have to be able to not only tell those leaders, 1170 00:46:28,178 --> 00:46:30,390 here are the answers to your questions 1171 00:46:30,390 --> 00:46:31,990 in the business mission, 1172 00:46:31,990 --> 00:46:35,505 but here's how our productivity and efficiency compare 1173 00:46:35,505 --> 00:46:38,360 against commercial sector benchmarks. 1174 00:46:38,360 --> 00:46:41,110 Where are we as good, where are we not as good, 1175 00:46:41,110 --> 00:46:42,750 and when not, why not. 1176 00:46:42,750 --> 00:46:44,620 You always have to be able to answer the question "Why?" 1177 00:46:44,620 --> 00:46:46,410 because often we're not as good, 1178 00:46:46,410 --> 00:46:47,260 when we're not as good, 1179 00:46:47,260 --> 00:46:49,990 it's often because we have specific strictures 1180 00:46:49,990 --> 00:46:52,720 laid on us from the legislature. 1181 00:46:52,720 --> 00:46:54,700 And when you do the root cause analysis, 1182 00:46:54,700 --> 00:46:56,310 you find those pretty quickly. 1183 00:46:56,310 --> 00:46:57,910 But we can quantify all of that. 1184 00:46:58,840 --> 00:47:02,530 Ultimately, the schemas, 1185 00:47:02,530 --> 00:47:04,340 the way in which we look at the data, 1186 00:47:04,340 --> 00:47:06,450 the specific data we look at, 1187 00:47:06,450 --> 00:47:09,943 goes right back to the hairball of IT, 1188 00:47:11,200 --> 00:47:13,970 but it's directed by those, 1189 00:47:13,970 --> 00:47:16,110 the questions the leaders need answers to. 1190 00:47:16,110 --> 00:47:17,750 Now, just like everything else, 1191 00:47:17,750 --> 00:47:19,940 those questions change over time. 1192 00:47:19,940 --> 00:47:22,132 The minute we give a useful set of answers, 1193 00:47:22,132 --> 00:47:23,900 they start to make changes, 1194 00:47:23,900 --> 00:47:26,580 and now they have a new set of data that they need, 1195 00:47:26,580 --> 00:47:29,520 or a new set of questions that we have to ask. 1196 00:47:29,520 --> 00:47:32,770 We're back to optimizing for change, as Nancy pointed out. 1197 00:47:32,770 --> 00:47:35,193 Did I answer the spirit of your question, sir? 1198 00:47:36,282 --> 00:47:37,115 - [Man] Sort of, yeah. 1199 00:47:37,115 --> 00:47:37,948 - Sort of? 1200 00:47:37,948 --> 00:47:39,837 Could I get a little-- - [Man] Well I mean like, 1201 00:47:39,837 --> 00:47:42,550 you know, you mentioned the financial systems, 1202 00:47:42,550 --> 00:47:44,633 you had a hard number. 1203 00:47:46,620 --> 00:47:48,660 300 unique financial systems, 1204 00:47:48,660 --> 00:47:51,632 and your goal is to get those down to one. 1205 00:47:51,632 --> 00:47:54,447 There's (mumbles). 1206 00:47:58,380 --> 00:47:59,660 I guess I'm looking more, you know, 1207 00:47:59,660 --> 00:48:02,710 are you looking to take measures 1208 00:48:02,710 --> 00:48:04,093 and lines of codes that you used. 1209 00:48:04,093 --> 00:48:06,643 Like BTL's code within your organization (mumbles). 1210 00:48:08,654 --> 00:48:11,810 Do you have like a more general hard numbers 1211 00:48:11,810 --> 00:48:13,350 that you're maybe trying to achieve 1212 00:48:13,350 --> 00:48:14,650 throughout the Department? 1213 00:48:15,600 --> 00:48:20,000 - So from the perspective of the size of the code base 1214 00:48:20,000 --> 00:48:20,833 of the Department, 1215 00:48:20,833 --> 00:48:23,430 or the size of the application base of the Department, 1216 00:48:23,430 --> 00:48:25,910 we don't even have good numbers to tell us 1217 00:48:25,910 --> 00:48:27,223 how big that is today. 1218 00:48:28,153 --> 00:48:31,923 And we've gone looking for this very deliberately. 1219 00:48:33,870 --> 00:48:38,110 Source lines of code, function points, use CASO's, 1220 00:48:38,110 --> 00:48:41,770 McCabe's Cyclomatic Complexity, you name the measure, 1221 00:48:41,770 --> 00:48:44,390 there's half a dozen to a dozen of them. 1222 00:48:44,390 --> 00:48:46,070 We have not been able to find them being 1223 00:48:46,070 --> 00:48:48,570 consistently captured or recognized anywhere. 1224 00:48:48,570 --> 00:48:50,720 So we're not attempting to use those. 1225 00:48:50,720 --> 00:48:53,480 As somebody who wrote code for a living for a long time, 1226 00:48:53,480 --> 00:48:54,990 and still writes code, 1227 00:48:54,990 --> 00:48:57,563 I mistrust most of those metrics anyway. 1228 00:48:58,910 --> 00:49:01,963 So I look at it at a much more macro level, 1229 00:49:03,070 --> 00:49:07,910 and this is purely arguable, right? 1230 00:49:07,910 --> 00:49:10,560 What's the definition of an application? 1231 00:49:10,560 --> 00:49:13,060 What's the definition of a system of applications? 1232 00:49:13,060 --> 00:49:14,660 What's the definition of a data center? 1233 00:49:14,660 --> 00:49:17,360 What's the definition of a piece of data or role? 1234 00:49:17,360 --> 00:49:18,797 These are highly arguable things, 1235 00:49:18,797 --> 00:49:20,300 and the more mature you get, 1236 00:49:20,300 --> 00:49:23,330 the more arguments you have about what they are, right? 1237 00:49:23,330 --> 00:49:25,310 This is a sign of the maturity in your space 1238 00:49:25,310 --> 00:49:27,260 that you're having those conversations. 1239 00:49:28,200 --> 00:49:29,450 So at that level, 1240 00:49:29,450 --> 00:49:34,450 I tend to mistrust many of the supporting measures. 1241 00:49:34,750 --> 00:49:37,040 I look more to macro level. 1242 00:49:37,040 --> 00:49:40,460 Let's assume that we generally know the right number. 1243 00:49:40,460 --> 00:49:44,442 A useful number of applications in a space. 1244 00:49:44,442 --> 00:49:49,300 In my world, where I came from, again, heretic, 1245 00:49:49,300 --> 00:49:51,640 the correct number of on-premise applications 1246 00:49:51,640 --> 00:49:56,023 for business functions in any large enterprise is zero. 1247 00:49:58,540 --> 00:50:03,160 Because you subscribe to softwares of service instead, 1248 00:50:03,160 --> 00:50:05,540 because business is business. 1249 00:50:05,540 --> 00:50:08,270 And there are best practices built into those solutions, 1250 00:50:08,270 --> 00:50:10,210 and if you move to those best practices 1251 00:50:10,210 --> 00:50:12,280 on softwares and service bases, 1252 00:50:12,280 --> 00:50:14,420 they keep them up to date from a functional perspective, 1253 00:50:14,420 --> 00:50:16,650 and they keep them up to date from a technical perspective, 1254 00:50:16,650 --> 00:50:18,570 and they keep them secure for you. 1255 00:50:18,570 --> 00:50:21,170 The thought that we're gonna do a better job 1256 00:50:21,170 --> 00:50:25,350 of implementing and operating business applications 1257 00:50:25,350 --> 00:50:28,453 on our premise is self-deluding. 1258 00:50:29,840 --> 00:50:32,030 We are too small. 1259 00:50:32,030 --> 00:50:34,940 We spend $46 billion a year on IT, 1260 00:50:34,940 --> 00:50:37,200 which is an astonishing figure. 1261 00:50:37,200 --> 00:50:42,200 The US IT industry is $1.7 billion, excuse me, 1262 00:50:42,477 --> 00:50:45,090 $1.7 trillion a year. 1263 00:50:45,090 --> 00:50:46,610 We're a rounding error. 1264 00:50:46,610 --> 00:50:49,310 We're never gonna be as good at managing IT 1265 00:50:49,310 --> 00:50:52,200 as the people who do it as a business. 1266 00:50:52,200 --> 00:50:55,490 And it is sensible for us to adopt those practices. 1267 00:50:55,490 --> 00:50:57,820 So in the business mission the correct number 1268 00:50:57,820 --> 00:50:59,680 of on-premise systems is zero. 1269 00:50:59,680 --> 00:51:02,433 In the war fighting mission, it's a different number. 1270 00:51:03,581 --> 00:51:04,414 All right? 1271 00:51:04,414 --> 00:51:06,757 In the intelligence mission, it's a different number. 1272 00:51:06,757 --> 00:51:09,010 But at a course grade level, 1273 00:51:09,010 --> 00:51:11,220 I wouldn't worry at all myself about the number 1274 00:51:11,220 --> 00:51:12,360 of sources lines of code, 1275 00:51:12,360 --> 00:51:15,220 or use CASO's, or function points, 1276 00:51:15,220 --> 00:51:16,630 of our on-premise applications 1277 00:51:16,630 --> 00:51:19,120 'cause we simply shouldn't have them. 1278 00:51:19,120 --> 00:51:20,920 We should move to a different model. 1279 00:51:21,990 --> 00:51:24,518 Which may be more in the spirit of your question. 1280 00:51:24,518 --> 00:51:25,750 I hope it was a useful answer. 1281 00:51:25,750 --> 00:51:27,470 - Let me just add, quickly. 1282 00:51:27,470 --> 00:51:28,660 At a macro level, 1283 00:51:28,660 --> 00:51:32,560 there's a very healthy discussion about metrics, 1284 00:51:32,560 --> 00:51:35,720 and driving change in order to increase efficiencies, 1285 00:51:35,720 --> 00:51:38,843 and to have savings across our systems, right? 1286 00:51:38,843 --> 00:51:40,440 And that's what's driving a lot 1287 00:51:40,440 --> 00:51:42,190 of these consolidation discussions. 1288 00:51:43,400 --> 00:51:48,040 But we need to maintain focus on metrics 1289 00:51:48,040 --> 00:51:50,530 that drive the case for change, 1290 00:51:50,530 --> 00:51:53,010 that increases capability, and increases security, right? 1291 00:51:53,010 --> 00:51:56,220 And I think that goes back to the theme of DOTUS this year. 1292 00:51:56,220 --> 00:51:59,420 And so we need to, we need to be cognizant of, as we, 1293 00:51:59,420 --> 00:52:00,420 as we develop metrics, 1294 00:52:00,420 --> 00:52:02,010 as we try to align with the leaders 1295 00:52:02,010 --> 00:52:04,729 that are pushing for this change, 1296 00:52:04,729 --> 00:52:08,400 that we're not exclusively driving towards 1297 00:52:08,400 --> 00:52:10,190 the efficiency conversation 1298 00:52:10,190 --> 00:52:15,190 without a proper emphasis on security and capability. 1299 00:52:16,900 --> 00:52:19,360 - Okay now the, I love the interaction, 1300 00:52:19,360 --> 00:52:22,427 so we're gonna shift to a question over here to my right. 1301 00:52:22,427 --> 00:52:23,590 - [Audience Member] Dave, first I'mma say, 1302 00:52:23,590 --> 00:52:24,620 this is all very exciting, 1303 00:52:24,620 --> 00:52:26,180 and if you take the long view, 1304 00:52:26,180 --> 00:52:29,010 this whole CDO thing is relatively new still, 1305 00:52:29,010 --> 00:52:33,300 and I wanna ask a question, a career path kinda question. 1306 00:52:33,300 --> 00:52:36,580 If you're a junior person in DOD who wants to aspire 1307 00:52:36,580 --> 00:52:37,930 to be a CDO, 1308 00:52:37,930 --> 00:52:39,330 what advice would you guys have? 1309 00:52:39,330 --> 00:52:40,900 I'd love to hear all four of you say that. 1310 00:52:40,900 --> 00:52:43,261 I'm askin' for a friend, okay, well, actually my daughter 1311 00:52:43,261 --> 00:52:46,440 (audience laughing) 1312 00:52:46,440 --> 00:52:50,516 who wants to look up and one day be a CDO of an agency 1313 00:52:50,516 --> 00:52:51,960 or piece of the Department. 1314 00:52:51,960 --> 00:52:53,720 You know, what kind of career advice would you have? 1315 00:52:53,720 --> 00:52:56,300 What kind of training, experience, or, you know, 1316 00:52:56,300 --> 00:52:59,881 any kind of career mentorin' you would give them? 1317 00:52:59,881 --> 00:53:00,714 - Who wants to go first? 1318 00:53:00,714 --> 00:53:01,840 Nancy, do you wanna go first? 1319 00:53:01,840 --> 00:53:03,460 - I'll take part of that. 1320 00:53:03,460 --> 00:53:06,740 This was probably not a position I expected to be in, 1321 00:53:06,740 --> 00:53:08,800 but if I look back on my career, 1322 00:53:08,800 --> 00:53:12,570 I had a range of experiences that really positioned me well. 1323 00:53:12,570 --> 00:53:16,076 Partly because I worked in roles in almost every 1324 00:53:16,076 --> 00:53:19,400 piece of the pie in that data life cycle up there. 1325 00:53:19,400 --> 00:53:22,360 On the mission side, on the business system side, 1326 00:53:22,360 --> 00:53:24,850 on the information management side, 1327 00:53:24,850 --> 00:53:26,780 as a long time IT program manager, 1328 00:53:26,780 --> 00:53:28,460 I started out actually as a user. 1329 00:53:28,460 --> 00:53:30,800 You're gonna find this a little bit of a funny story. 1330 00:53:30,800 --> 00:53:32,160 When I got mono in college, 1331 00:53:32,160 --> 00:53:34,610 I dropped my computer science class and undergrad. 1332 00:53:34,610 --> 00:53:36,390 I was a Liberal Arts major. 1333 00:53:36,390 --> 00:53:38,830 But then I got a Master of Science in Information Systems, 1334 00:53:38,830 --> 00:53:41,140 so I'm a little bit of a hybrid. 1335 00:53:41,140 --> 00:53:43,780 That's really been very effective here. 1336 00:53:43,780 --> 00:53:45,290 One of the strongest things, I think, 1337 00:53:45,290 --> 00:53:47,950 is building some domain knowledge of the organization 1338 00:53:47,950 --> 00:53:49,350 you're going to be working with, 1339 00:53:49,350 --> 00:53:50,670 building partnerships, 1340 00:53:50,670 --> 00:53:53,470 and building those engagement and collaboration skills, 1341 00:53:53,470 --> 00:53:55,620 'cause any of the specific technical things, 1342 00:53:55,620 --> 00:53:57,760 and actually one of my professors in grad school said this, 1343 00:53:57,760 --> 00:54:00,830 the half-life of your degree at the time was a couple years, 1344 00:54:00,830 --> 00:54:03,060 now I think it might be a couple months. 1345 00:54:03,060 --> 00:54:05,240 Because the technology that some of you are doing 1346 00:54:05,240 --> 00:54:07,890 such great innovative things is changing. 1347 00:54:07,890 --> 00:54:10,280 But sort of that breadth and range of experience 1348 00:54:10,280 --> 00:54:13,940 of living on the mission side and the service provider side, 1349 00:54:13,940 --> 00:54:17,180 that I think has made me stronger in the CDO role. 1350 00:54:17,180 --> 00:54:19,180 At least that's been part of my journey. 1351 00:54:20,150 --> 00:54:21,700 - John, do you want to? 1352 00:54:21,700 --> 00:54:22,533 - I agree with that. 1353 00:54:22,533 --> 00:54:25,200 I mean, there's a difference between meaning and doing, 1354 00:54:25,200 --> 00:54:29,690 and you need the healthy amount of knowing how to do, 1355 00:54:29,690 --> 00:54:33,630 but especially in the circles that we're running here, 1356 00:54:33,630 --> 00:54:34,463 and all of you, 1357 00:54:34,463 --> 00:54:38,090 this is a very broad and complex set of organizations, 1358 00:54:38,090 --> 00:54:42,470 and so having the skills to be able to interact 1359 00:54:42,470 --> 00:54:45,690 and drive change with this amount of complexity 1360 00:54:45,690 --> 00:54:49,973 is something that requires deliberate thought. 1361 00:54:53,490 --> 00:54:55,470 - I think I'd start by acknowledging 1362 00:54:55,470 --> 00:54:58,870 that the role of Chief Data Officer is different 1363 00:54:58,870 --> 00:55:00,290 in different organizations, 1364 00:55:00,290 --> 00:55:02,824 and there are a couple of major dimensions of that. 1365 00:55:02,824 --> 00:55:05,180 The more mature an organization is, 1366 00:55:05,180 --> 00:55:07,700 with respect to data management, and analytics, 1367 00:55:07,700 --> 00:55:11,500 and with respect to its data culture, 1368 00:55:11,500 --> 00:55:15,410 the more likely the role is to be a general manager role, 1369 00:55:15,410 --> 00:55:20,233 rather than a role in which one leads by example. 1370 00:55:21,208 --> 00:55:22,690 Those are really different. 1371 00:55:22,690 --> 00:55:25,880 I've seen people in very mature organizations 1372 00:55:25,880 --> 00:55:29,150 who are successful CDO's who are general managers 1373 00:55:29,150 --> 00:55:31,710 who haven't been hands-on with the technology 1374 00:55:31,710 --> 00:55:33,163 in quite a while. 1375 00:55:35,004 --> 00:55:35,837 And they can do it, 1376 00:55:35,837 --> 00:55:38,310 because they have so many people in their direct control 1377 00:55:38,310 --> 00:55:42,070 with a large budget that they can rely on those people 1378 00:55:42,070 --> 00:55:43,370 to get things done. 1379 00:55:43,370 --> 00:55:46,590 I happen to think that that's a place 1380 00:55:46,590 --> 00:55:48,295 I'd never want to work, but, 1381 00:55:48,295 --> 00:55:51,110 but some of them make it work, right? 1382 00:55:51,110 --> 00:55:53,663 In other organizations, where you're much less mature, 1383 00:55:53,663 --> 00:55:58,663 where you have decentralized control over resources, 1384 00:55:58,771 --> 00:56:01,950 not only with respect to data, but with respect to IT, 1385 00:56:01,950 --> 00:56:04,140 with respect to decision making, 1386 00:56:04,140 --> 00:56:06,900 then you have a need 1387 00:56:06,900 --> 00:56:08,490 to lead by example. 1388 00:56:08,490 --> 00:56:11,066 You have a need to be a hands-on practitioner, 1389 00:56:11,066 --> 00:56:15,050 you can't let yourself get far away from the advancements. 1390 00:56:15,050 --> 00:56:16,040 And this is a challenge, 1391 00:56:16,040 --> 00:56:19,310 because A, you need that domain knowledge, 1392 00:56:19,310 --> 00:56:22,600 and you're not gonna be proficient in every domain. 1393 00:56:22,600 --> 00:56:24,850 If you're proficient in the cyber domain, 1394 00:56:24,850 --> 00:56:26,390 that's all you do. 1395 00:56:26,390 --> 00:56:28,960 If you're proficient in financial management domain, 1396 00:56:28,960 --> 00:56:30,697 that's all you do. 1397 00:56:30,697 --> 00:56:33,496 If you're, you know, pick the domain. 1398 00:56:33,496 --> 00:56:36,750 To be proficient, to understand the data structures, 1399 00:56:36,750 --> 00:56:39,890 the terms, the meta-data, the measures of success, 1400 00:56:39,890 --> 00:56:41,160 the specific algorithms, 1401 00:56:41,160 --> 00:56:43,516 and every domain has specific algorithms, 1402 00:56:43,516 --> 00:56:45,700 if you're not in that space all the time, 1403 00:56:45,700 --> 00:56:47,320 you probably are not current. 1404 00:56:47,320 --> 00:56:49,930 You need math and stats knowledge, 1405 00:56:49,930 --> 00:56:52,150 this is some fundamental things you have to be able 1406 00:56:52,150 --> 00:56:53,820 to understand and apply, 1407 00:56:53,820 --> 00:56:56,823 and you need a set of computer science skills as well. 1408 00:56:57,670 --> 00:57:00,720 I'm not joking when I talk about the first question 1409 00:57:00,720 --> 00:57:03,530 I ask a data scientist candidate is, 1410 00:57:03,530 --> 00:57:05,273 do you code in Python or R? 1411 00:57:06,640 --> 00:57:08,460 Because if you're not coding in one of those, 1412 00:57:08,460 --> 00:57:09,750 actively coding in one of those, 1413 00:57:09,750 --> 00:57:11,200 you're not a data scientist. 1414 00:57:11,200 --> 00:57:12,670 You may be a valuable resource, 1415 00:57:12,670 --> 00:57:15,095 but you're not a data scientist. 1416 00:57:15,095 --> 00:57:17,170 But you also need soft skills. 1417 00:57:17,170 --> 00:57:18,940 You need the relationship skills, 1418 00:57:18,940 --> 00:57:20,420 and you need business savvy, 1419 00:57:20,420 --> 00:57:23,430 you need the ability to communicate effectively 1420 00:57:23,430 --> 00:57:24,810 in writing or in person, 1421 00:57:24,810 --> 00:57:27,610 and you need the ability to facilitate. 1422 00:57:27,610 --> 00:57:30,450 All of those things are also critical to your success. 1423 00:57:30,450 --> 00:57:33,050 Now what we do is a team sport. 1424 00:57:33,050 --> 00:57:35,140 So no one person has everything. 1425 00:57:35,140 --> 00:57:37,220 That's not what we look for. 1426 00:57:37,220 --> 00:57:41,560 But we look for a lot of those things in any one person, 1427 00:57:41,560 --> 00:57:44,050 and hope that the team's mix of strengths 1428 00:57:44,050 --> 00:57:46,463 and weaknesses overall balances out. 1429 00:57:47,640 --> 00:57:49,210 So part of the answer is, 1430 00:57:49,210 --> 00:57:51,290 it depends on what organization you're going into. 1431 00:57:51,290 --> 00:57:54,400 I know some peers in the federal government space 1432 00:57:54,400 --> 00:57:58,110 that like me are about leading by example, by doing, 1433 00:57:58,110 --> 00:57:59,170 by creating, 1434 00:57:59,170 --> 00:58:04,170 as well as by functionally being a force multiplier. 1435 00:58:04,570 --> 00:58:05,403 Right? 1436 00:58:05,403 --> 00:58:06,760 'Cause you can't do all of the work. 1437 00:58:06,760 --> 00:58:09,380 So you have to show other people how to do that work. 1438 00:58:09,380 --> 00:58:11,680 And then I know some where it's purely 1439 00:58:11,680 --> 00:58:15,970 about being a general manager or setting policy, 1440 00:58:15,970 --> 00:58:20,810 and so the challenge comes to one of those might appeal 1441 00:58:20,810 --> 00:58:22,200 more than another, 1442 00:58:22,200 --> 00:58:24,240 and then you have to select the right organization 1443 00:58:24,240 --> 00:58:27,010 and the right boss if you're gonna be successful 1444 00:58:27,010 --> 00:58:28,877 in that role of CDO. 1445 00:58:30,710 --> 00:58:32,030 - And I would just add to that. 1446 00:58:32,030 --> 00:58:36,740 So technical skills is a basis and a foundation, certainly, 1447 00:58:36,740 --> 00:58:41,410 R, Python, statistics, computer science skills. 1448 00:58:41,410 --> 00:58:42,640 In my role, 1449 00:58:42,640 --> 00:58:45,410 one of the things I've learned about my role out as a CDO 1450 00:58:45,410 --> 00:58:47,350 over the last nearly 12 months, 1451 00:58:47,350 --> 00:58:50,860 is I consider myself almost like a Chief Marketing Officer. 1452 00:58:50,860 --> 00:58:52,280 And that's a translator. 1453 00:58:52,280 --> 00:58:53,620 I have to be able to translate. 1454 00:58:53,620 --> 00:58:56,240 I have to be able to translate the value that a CDO 1455 00:58:56,240 --> 00:58:58,820 function and the things that we endeavor to do 1456 00:58:58,820 --> 00:59:01,350 have relevance and meaning to our mission partners. 1457 00:59:01,350 --> 00:59:03,840 Because they look at us, quite honestly, as overhead. 1458 00:59:03,840 --> 00:59:07,060 Everyone says data's important, data's really important, 1459 00:59:07,060 --> 00:59:09,830 but when it comes to the actual work related 1460 00:59:09,830 --> 00:59:12,530 to really shore up your data holdings, 1461 00:59:12,530 --> 00:59:16,457 understand them, and treat them as an asset, 1462 00:59:16,457 --> 00:59:18,970 people often times take shortcuts. 1463 00:59:18,970 --> 00:59:21,380 So with that, I think we're out of time. 1464 00:59:21,380 --> 00:59:24,530 I would offer that if you would like to have additional 1465 00:59:24,530 --> 00:59:26,240 conversation with any of our panel members, 1466 00:59:26,240 --> 00:59:28,551 please catch us at the end of the session. 1467 00:59:28,551 --> 00:59:31,600 This hour has gone by lightning fast. 1468 00:59:31,600 --> 00:59:33,840 I think this could have easily been a two hour session, 1469 00:59:33,840 --> 00:59:35,900 and we probably still would've run out of time. 1470 00:59:35,900 --> 00:59:37,660 Thank you so much for your attention, 1471 00:59:37,660 --> 00:59:40,043 and I'd like to give a round of applause 1472 00:59:40,043 --> 00:59:41,930 (audience applauding) to our panel members.