DFIR Expert Interview: Kimber Dowsett

Please introduce yourself and tell us what you do, and what your company does.

I’m Kimber Dowsett and I’m a Director at Krebs Stamos Group (KSG). We conduct  cybersecurity consultancy engagements for high-profile organizations that may or may not have experienced a high-profile breach or acquisition, or simply want a world-class assessment of their org’s overall security posture. Our clients span multiple industries and our engagements produce outstanding outcomes with a diverse set of deliverables. 

I’m always interested in how people came into cybersecurity as a career. It seems like no two paths are the same. What did you start out wanting to be? Did it look like cybersecurity, and if not, how did you find your way into the field? 

I went to graduate school for fine arts. I have an MFA in mixed media and new media theory, so no, cybersecurity was not on my radar. I was recruited out of graduate school by Apple to be part of a new “Creatives” program launching in their retail stores. They wanted staff who could train Apple customers on their Pro Apps like LogicPro, Adobe Creative Suite, etc. I flew out to Cupertino for training and on the very first day, I saw people in another training room tearing apart computers with screwdrivers. It looked a lot more interesting than playing with Photoshop so I said, “I want to do that. What are they doing?” My instructor explained that it was the Mac Genius program training so I said, ”okay I’m gonna go do that instead.” I was told no since I’d been flown out for the Creative program. During the lunch break, I scooted over to the other training and politely sat myself. No one knew what to say, so I stayed in my new seat and completed Genius training. 

I was, I believe, one of the first few women who successfully passed all of the Apple certification exams following that training: OSX certification, desktop repair, and laptop repair. At the time they Apple their trainings into multiple certifications. I returned to the east coast working in Apple retail stores and stayed with them a few years before moving on. Initially I worked as a Genius, but quickly moved my way up through the hierarchy to become a regional Lead Genius mentor for the east coast. I traveled up and down the east coast training lead geniuses on techniques to lead better teams, improve time-to-repair statistics, and increase success rates with hardware repairs. At the time, Apple wasn’t outsourcing laptop or desktop repairs and completed all repairs in the back rooms of Apple retail stores. I also trained Mac Geniuses on fundamentals of operating systems, how to troubleshoot and look for signs of malicious activity on OSX and, at the time, OS9 as well (that dates me, doesn’t it?).

Was that your introduction to UNIX?

I guess you could say my first introduction to UNIX systems was in college since the art department used Macintosh machines and OS9 was “UNIX-like”. I really didn’t start using the command line on the regular (unless BBS and IRC count) until I started with Apple in 2005. There were some retail stores, about a hundred or so when I started. Locally, we were doing our own in-store network configurations, following the instruction book on how to secure wireless, and adapt if/when it was compromised. All of our networks were managed through a UNIX box without a GUI. It was up to Mac Geniuses to figure out the attack vectors in their local stores. I spent a lot of time at the Pentagon City store across the street from the Pentagon, so we had to tighten up our network a little more than some of the other retail locations in our area since we serviced a lot of government personnel.

When I was ready for something else, I moved on from Apple and ended up at a startup that was eventually acquired by the Economist. I was performing what we called “QA engineering” but I was doing SQL injections and helping engineers sanitize application input. We were acquired by the Economist who brought in a new CTO who asked me what I did and, once I explained it,  he responded, “ you’re not a QA engineer, you're a security analyst” - and just like that, a job title changed the trajectory of my career. I’d been doing security at Apple but we didn’t call it that; we just labeled what we did it “troubleshooting” or “locking down networks.” At the startup, I was referred to as a Sr. QA engineer until a CTO changed my title to a security analyst, ultimately shaping the next 15 years of my career. Job titles matter.  As a security analyst, I was able to get my next gig with a company that was also acquired. I was able to do threat hunting, malware analysis, and work on securing our proprietary hardware, known as transponders, which was a listening device used to calculate radio ratings. We were acquired by Nielsen (TV ratings), so there was a running joke amongst my colleagues that if I worked for a company, the company was acquired. After the Nielsen merger and spending time working with the transition team, I made the decision to join NASA as a senior mission information specialist, securing instrument and ground systems, and conducting network security assessments. The instrument and ground systems I secured controlled the cryogenics, optics, and lasers used to ensure systems would function correctly following a launch into space. It was cool to work at NASA but I was always perplexed by the number of nation-state adversaries who wanted access to our data since nearly all of it was public. We made sure our systems were air-gapped so attackers couldn’t pivot all over the place. The Solaris 9 systems weren’t awesome to work on and it was a big deal when upgraded a couple of our mission systems from Solaris 9 to 10. Some of the systems made it all the way up to S13 by the time I left, but I think that was about as high as we could go without risking the integrity of the hardware. 

It was fun working at NASA for the prestige (and swag), though the day-to-day was a bit difficult as I was working on legacy systems, mainly Solaris9, WindowsXP, and Windows7 systems. I saw my first WindowsNT system in production there, attached to something in space so that was pretty scary (but also super fun). 

I’m intrigued by your handle: Mz Bat. Is there a story behind it?

In the early days of hackers hanging out on IRC, people who presented as female would put a Mz in front of their name so other female-presenting hackers could identify fellow non-males. We’d pick names or animals we liked, and stick a Mz on the front. Since my friends called me Kimber Bat (because I was so goth that bats flew out of my mouth when I yawned), my IRC handle mz. bat was born. For a chunk of the 2000s, I DJ’d under that name pretty regularly, and more recently, I’ve shown up as the guest DJ for “Line Con” at DEF CON a time or two. 

It wasn’t tied to my real name until 2016 when the feds outed me in a (positive) news story. That was the first time there was a connection. They wrote that Kimber Dowsett spoke on an Access Now panel about privacy and encryption. They put my Twitter handle in parentheses and that was it. I’d managed to keep things separate for a long time, about 15 years before my name was tied to my handle. C’est la vie, I suppose. OpSec isn’t eternal. 

What would you say is a low-hanging fruit for enterprises to improve their security posture? We talk to a lot of people in DFIR about when they first go into a place and think, this is horrible, everything is wide open. What do you think is the most common gotcha you see out there?

User assets. Whether they're corporate issues managed by an MSP, whether they're BYOD, or worst-case scenario, they're not managed by anything.

I acknowledge that home networks add another attack vector, especially now that a lot of home networks have become work networks as well. Know your threat model. For me, even at a high-profile company I feel amply protected at home with a firewall and segregated networks. (That’s not a challenge, hackers.)

I think asset management tends to be the easier attack vector, especially if users are skimming through their security training and don't really understand what a phishing email looks like. If they click a link on their work computer, they may not understand they’ve just opened up their work asset and potentially their home computer or other devices on their home network to attackers.

With proper training and a robust security education program, companies can avoid some pain points by treating users as humans susceptible to sophisticated phishing campaigns and attacks. It’s not a user’s fault if they fall for attackers’ tricks, especially if there wasn’t a solid program in place to teach them what to inspect before clicking a link (like email headers). I think on a technical level, you get what you pay for so it’s not enough to deploy an enterprise solution; it’s imperative to have professionals on staff (or contract an established security consultancy like KSG) to ensure the selected enterprise solution is deployed correctly based on threat modeling and organizational risk. Something as simple as upgrading an org’s Microsoft offering from E3 to E5 to extend log retention and improve user management and administrative controls can have an immediate impact on an organization’s overall security posture. A lot of the things you see in Enterprise IT related to configurations are tied directly to the package an organization is paying for, so it’s important to have a technical set of eyes on contracts related to Enterprise IT solutions.

A common issue I see within cloud infrastructure deployments is administrators logging in as root, removing the ability to audit changes.  User management is almost always one of my findings in an initial assessment. It’s rare that companies have it buttoned up the way they should. Another common finding is related to asset management and BYOD that aren’t managed by an MSP. Also important to note is that sometimes I see companies relying on their MSP to provide security, but haven’t included specific controls they want/need enabled in their vendor contracts. On the MSP side, there’s a person selling an enterprise solution to someone on the org’s enterprise side who says “ok, we’ve bought this solution, we have a contract so we’re secure” and wrap up for the day. On the org’s enterprise side, they may have no idea that every user in their org’s fleet is an admin until a security professional completes an assessment months to years later. The org’s representative didn’t know what they needed to negotiate in the contract with their MSP. A scenario like this can be mitigated by having a security professional help out, consulting with both the org’s representative and the finance team when it’s time to negotiate an MSP contract. This helps ensure that necessary controls will be enabled and paid for. 

That is some really good advice

What’s the MSP’s responsibility for incident response? Do they have the ability to kill access to your fleet? On the enterprise side, they may not know what they need and have no security person to advise them. They may only know to check the compliance box to push updates. That's good and necessary,  but there are hundreds of other things that need to be implemented by the person or team managing an enterprise fleet. 

As a consultant, I like to point out the great things an organization is doing and tie them to changes that could make things better. I prefer an approach like, “you’re doing well here and I can see that you’re thinking about this. That’s great! I did notice that you’re only retaining logs for 90 days and your monitoring and alerting could use a little work. It’s good you’re retaining logs, but what are you doing with them? Who is looking at them? What if I told you that most organizations don’t discover they’ve been breached until after 90 days have passed? By not retaining logs a bit longer, you risk losing evidence of a breach without any way to see what was accessed. A breach is never a great thing to be in the news for, but it’s worse if you have no way to investigate what happened.” I’m not a fan of security folks who talk down to clients. I prefer to consult with empathy and lead with kindness.

You founded the Mock Interview and Resume Review (MIRR) workshop which seems like a really cool initiative. Can you tell me more about that? 

Of course. It’s on GitHub and the framework is open-source, meaning anyone can contribute - so definitely feel free to go and do that! The workshop started as a way to give underrepresented and marginalized communities mentorship as they prepared their resumes or prepped for interviews. The goal has always been to help folks who are unemployed or underemployed get their foot in the security door or move up in the cybersecurity field. 

I was conducting resume reviews and helping folks prepare for interviews, giving priority to folks from marginalized communities. Lesley Carhart, aka  hacks4pancakes, was doing a similar thing for resumes for veterans because they were a reservist who had a passion for helping vets. (They just retired after 21 years of military service, so kudos and congrats to Lesley and thank them for their service.) Lesley and I ended up partnering about 7 years ago, providing mentorship and guidance to anyone who signed up to chat with us. We combined forces and ran our first public offering at a security conference, DerbyCon 2015. Over the years, we’ve created sample technical questions and sample culture fit questions, and a resume framework to help folks get noticed by potential employers. We recognize that there are folks who’ve been in the field for a long time who’ve never heard of culture fit questions, so it’s great to give folks a sense of “here's what to expect” in an industry interview. The workshops are only successful because of volunteer participation. We invite volunteers who are hiring managers or high-level C-suite folks who have been in the cybersecurity field for a long time. This isn’t a place where we let people come to practice interviewing other people. Our volunteers come in as mentors for an hour and it’s not unusual for us to hear that a participant’s experience with their mentor literally changed their life.

Before covid hit, we ran our workshop at the final DerbyCon in 2019, We were excited to announce that we hit the milestone of 100 folks who’d directly reached out to tell us they had found jobs or received promotions as a result of attending the MIRR workshop. We couldn’t have asked for a more positive outcome. I’m proud of that workshop and the people who volunteer their time and participate in conferences. It’s on GitHub so folks can take it to their local security conferences, BSides events, or career fairs, or use it to go out into the world and volunteer their time locally. Lesley and I don’t want to own this thing forever and I’m excited the framework has been used at PiCon, multiple Bsides events, and as a speaker, I was able to champion MIRR internationally at  BruCON and was thrilled to see that BruCON runs a similar mentor/mentee program at their event. I’m so honored to be part of a community of folks who are willing to volunteer their time to help others.

We do want to do more stuff out in the community and this might be a good thing to center it around.

Definitely. Get people out there, even if it’s just a conversation for half an hour. At MIRR, volunteers come for an hour and see two people, two sessions. We do have people who book two sessions and want a resume review for the first half and an interview for the second. It is not a hiring or recruiting event since that puts way too much pressure on participants; it’s a learning experience. It’s also good for volunteers, especially those who haven’t had a lot of exposure to working directly with marginalized communities. It gives folks an opportunity to really understand the challenges others can face in the interview process due to unconscious bias or not knowing how to represent experience on resumes. Little nudges from professionals who know hobbies are relevant and can be professionally represented on resumes is valuable guidance for folks trying to enter the cybersecurity field. For example, entry-level folks should include interests in Hack the Box or other CTFs on their resume. It shows initiative. Drive is not something you can instill or teach someone, you either have it or you don’t. If relevant interests are listed on an otherwise short resume, it increases the chances of a candidate having an opportunity to discuss with their interviewer. 

What are your thoughts on the current cybersecurity vendor market?

I believe there are a lot of good companies out there and I think there are a lot of charlatans. I’d say do your research. As with any field that has the potential for folks to make a lot of money quickly, there are new cybersecurity business ventures popping up all over the place. Due diligence around who the founders are and where their funding is coming from is important. That’s probably more about vetting vendors, but there are a lot of them out there. I'm always fascinated to see vendors advertise or have marketing campaigns based on things they've done or what they’re doing in the security space when my colleagues and I have never heard of the company or their founders. At this point, I feel like I have a pretty good idea of who is out there and what they’re doing. If you have doubts about a vendor or can’t find any real information on them, I’d say keep shopping around.

Who is in the business of trading stock rather than value or solving problems…

It’s a lot like contractors for your house, if they tell you they can start in 2 weeks and their price is 25% below the contractors who have a 3-month wait, it’s probably a red flag. We need something like Angie’s list for contractors for cybersecurity vendors. We’ve almost reached critical mass for vendors and your average person who is responsible for making those decisions at an enterprise organization might not understand how to do due diligence. They may just meet with the contractor or founder and get a good vibe, but we just can't do business like that anymore. It’s possible the vendor has a good vibe because they’re a good snake oil salesperson who knows how to wow people, so let’s all do some background checks and vetting before signing any contracts.

Have you always had an understanding of how serious the implications of cyberwarfare are or was there a holy shit moment when you realized it was much more than lulz or financial crime? For me it was Stuxnet.

I don’t think I had a full understanding until I joined the federal government, and then realized that APTs could make satellites fall out of space if they wanted to. That’s pretty scary. Stuxnet for sure, and others, like Heartbleed - the way that APTs were able to use some of these massive vulnerabilities is terrifying. 

Heartbleed was out in the wild for years before anyone noticed it…

And exploited for how long before anyone spotted it? My interests are now in exploiting the CI/CD pipeline. If you can own the pipeline, game over. I don’t know if I’d sleep very well if I was a security engineer for one of the major CI/CD SaaS products or vendors. 

I think Jenkins, Circle, Travis, and Github are in danger of being owned now. I’m interested in how to break them or break into them, but I (probably) wouldn’t want to be on the hook for securing them. As of the date of our first chat, many of those teams are doing a great job but I don’t envy the stress levels of their security teams. 

EDITORS NOTE: This interview took place before the disclosure of the recent Github hack.

I’m a fan of building stuff with microcontrollers and I noticed on your LinkedIn you have the same interest. What is the project you’ve built that you’re most proud of?

I like to do PCB projects, the kind you see on electronic blinky badges at conferences. I have a bat badge out there in the wild somewhere. I designed the art and hand soldered LED lights on 200 badges for DEF CON and tortured my fingertips for another run of 200 for (now retired) DerbyCon.

The badge itself didn’t do much. It had blinky lights with a little button that changed the light pattern and if you plugged it into your computer, it had a few crypto challenges. It was tiny and simple and I think that’s why it did so well. I did see one was auctioned on eBay for charity by attrition.org recently and that was great. 

You list yourself as a gamer so I have to ask. What’s your favorite video game?

Left4Dead and Left4Dead2. But now that I have a ps5, Back4Blood is like L4D2 on steroids. Soooo good! Back in the day, I loved Portal and I probably lost a chunk of the 2000s playing Halo. 

Any thoughts about the cybersecurity community in general?

I think we’re doing a better job with inclusivity, but I think we still have a long way to go to learn about unconscious bias. We also need to work harder as a community to be more welcoming to folks who are curious and want to learn more about security. For example, if you have creators who are genuinely curious about NFTs and how to not get scammed, telling them not to participate is not effective. I’d rather say “here are the risks and here are the things to look out for” and focus on education rather than negativity. 

Gatekeeping has been rampant in general, but I see it changing.

Are there any leaders or community members that you look to for inspiration or who inspired you along the way?

Lesley Carhart is right up there. Amelie Koran. Black Girls Hack is a great organization, led by Tennisha Martin. Black Girls Hack has regular Sunday meetups to have folks come and speak to women who are part of their community, offering the chance to ask questions about being a professional in the cybersecurity field. 

Is there anything that keeps you up at night?

Not really, unless I’m actively involved in an engagement. The things that keep me up at night are more related to world events and human suffering. 

In cybersecurity, I have a litmus test. Is it life or death? Is something going to fall out of space? Are we going to lose a client that could mean having to release employees? But day-to-day I don’t stress about AD environments or substacks or cloud infrastructure anymore - BUT I’m not minimizing what might feel detrimental or stressful to responders working in those spaces. 

In most cases I prefer to take a step back and ask, is this life or death or a global game-changer, or perhaps a mortal danger to our company? If I can say no to those three, I can take a beat, process what’s happening, and operate from a place of thoughtful incident response rather than reactionary response where I’m most likely to miss something. As cybersecurity professionals, many of us have to respond to incidents that are really stressful, but if we can take a step back and look at our environments in general and proactively prepare solid incident response plans for everything from ‘our database has been breached’ to ‘aliens invaded Washington, DC’, it becomes a little less stressful to respond to an incident that’s in between.

Also, the first plans you develop may be a little bit wrong, but it’ll be okay. Iterate. 

Do you have any advice for people early in their careers considering cybersecurity? 

If they’re considering going into cybersecurity, they should talk to people already working in the field. If they’re already employed somewhere that has a security team, those are great folks to talk to. Explore internships or fellowships that allow a tour of duty to try different areas of cybersecurity. When I came up, you couldn't get a degree in cybersecurity; it didn’t exist. There was computer science and that was it. We didn’t call it security, but rather we called it quality assurance (QA) or operations (Ops). There are programs now for a few subsets of cybersecurity, from privacy to IT/Risk controls to computer science (CS) with a focus on secure code. Cybersecurity as a field and an area of study is huge now. It's a matter of figuring out what interests you, but if you don’t know what areas to explore, it’s a good idea to look for opportunities to learn or be mentored. 

SANS courses are expensive but they have a work-study program that offers tuition assistance for folks selected to work as a teacher's assistant for various courses. If accepted, participants attend the course and receive online materials and exam attempts (if applicable) for a fraction of the full-price cost of the course. It can still be expensive, but if you’re working somewhere with small training budgets, it’s a great way to utilize funds.

There’s no substitute for research into different areas of cybersecurity, taking time to assess what you’re good at, and finding an area of the field that is both interesting and plays to your strengths. If you happen to see the MIRR workshop pop up on a security conference schedule near you, it’s free for attendees. Sometimes we’re even parked in a lobby making it free for anyone who signs up. There are also career fairs and social media as options to make connections, but do your best to talk to people who are actually doing the job you want to do. In a lot of cases, any given cybersecurity job or role may not be what you think it is.

Any predictions for the future?

I think we’ll see an uptick in attacks on the CI/CD pipeline and in geopolitical-based cyberwarfare. We’re seeing it now and I think it gets worse, not better. I try not to be doom and gloom though. On a positive note, I think the community and the folks employed in cybersecurity roles are becoming more representative of the population, and areas of focus within cybersecurity are becoming more refined so folks can receive more training and mentorship in those roles. 

My hope is that we come to appreciate the generalist as much as the expert who has very focused attention and knowledge base. I think there is space for both. There is a great book called Range: Why Generalists Triumph in a Specialized World (David Epstein, 2019). It addresses the issues surrounding the question, “What if you spent your life as an expert in a field that is suddenly rendered obsolete?” A decade ago we didn't see the shift to cloud coming; we were still installing server racks at data centers in Reston, Virginia. Most of us who have been in the field for 10+ years have had to adapt to migrations to 3rd party cloud providers and learn how to secure our environments all over again (infrastructure as code). The fundamentals of infrastructure haven’t changed but the tools sure have. 

For folks whose day-to-day job may have been something like shell scripting on a controller laptop in a data center, in many cases, they either adapted, retired, or shifted careers to highly valuable tech-adjacent roles, like technical project managers (TPMs). The latter has become a staple in many cybersecurity and technical firms and we’ve been fortunate to have a wave of TPMs help our orgs hit milestones and deliver great work because they have a solid foundation in SDLC. 

Did I ever think as a Perl developer that it wouldn’t be the thing we always used? When one of the startups I worked for was acquired, our parent company wanted us to rebuild everything we’d built in Perl earlier as widgets in Java. Was I suddenly supposed to go learn a new language? Also, people like me who were doing “QA” in the early 2010s knew that Java was, at the time, one of the more vulnerable languages on the planet. That said, it persists and there are some kick-ass Java programmers out there. I can’t build stuff in Perl anymore (mainly because nobody wants it), and I do hear the occasional complaints when we have reviewed something written in PHP (rightfully so). I had to learn just enough Python to make my microcontrollers work, but don’t put me on a project that needs a ton of code because that’s just not me anymore - and that’s ok. I adapted.  

This has been super awesome, and I appreciate you taking the time to meet with me. 

It was so nice to meet you, too. Thanks for reaching out!