Tramscript
Mark:
Welcome to People Tech, the podcast of the HCM Technology Report. I’m Mark Feffer. My guest today is Dan Finnigan, the CEO of Filtered. Their platform evaluates candidates with brief, comprehensive job simulations, all customized to an employer’s tech stack. We’re going to explore the changing role of job simulations, what candidates think about it, and more on this edition of People Tech. Hello, Dan. It’s good to see you. What does Filtered do? What’s the value proposition?
Dan:
Well Filtered allows companies to interview candidates all over the world with a rich, automated interview that simulates the job and therefore simulates the skills one would do on the first day of the job. In other words, a lot of the work we do today is cloud-based. Filtered focuses on now job simulations that are in the cloud. We started off as a coding simulation platform, targeted towards engineering hiring, and we then moved into data science simulations and therefore the hiring of data scientists. We recently launched an overall job simulation platform for any job on the cloud, first focused on DevOps job simulations. When you think about what happens on the first day of the job, a new employee gets a laptop configured with the tool sets that they will use on the job. We simulate that, we record it, and we enable hiring managers to ask questions and video record the answers, so that candidates can explain their work. And we have fraud detection capabilities that enable the company to verify that the person is who they say they are.
Mark:
You’re talking about job simulations. Do they really work? Are they true simulations or are they just in a little box that’s like a real day on the job?
Dan:
No, they really do work. The first one we did was to take an IDE cloud-based coding environment, but enable it to be configured to any of the languages and tool sets that an engineer would use on the first day of the job. It was focused only on coding simulations, because we felt very strongly that coding tests don’t really assess one’s coding skills. We also believe that recruiting was overly reliant on people’s resume and their job history. There is a blog post, I should get the name of it, of a Silicon Valley CTO who said that the only way to interview an engineer, a software engineer, is to watch him or her code for an hour. And Filtered foresaw that need and built a completely configurable IDE coding environment and auto scores the output, meaning you’re asked to create software that can actually run and will score the results relative to the goals of the hiring manager, what tests they want to have verified in the results.
What we’ve done since then is, we built a live version of this so that when an engineer does coding work, it’s all recorded, they’re auto scored on the output, but lots of other signals and information is also recorded that you can go back and look at. How often did they run tests? Did they use code from somewhere else on the web to help them out, maybe their GitHub or did a search? All that’s recorded. What we created after the pandemic because some of our customers said, “I can’t bring anyone into the office anymore.” We created a live version, so that the hiring manager and the candidate could get into the same virtual room, with all of the recorded work done by the candidate, poured it into the room, a room that has a whiteboard you can draw on together.
But more importantly, where the hiring manager and the candidate can code together, and the hiring manager can talk about, “I see how you did this. We would’ve preferred if you had tried something like this.” In the act of doing that, like a lot of new technology startups, we had an epiphany and realized that we’d essentially created a terminal experience, a cloud terminal experience, and that’s when we decided we should do this for any cloud-based work. And that’s when we started first focused on DevOps.
You asked, “Is this the real day on the job?” We have for one company, have been working with them, on creating a cloud interview using their own robotic operating system, that engineers use at some point when they get into the job. For 66 Degrees, which is the largest provider of Google Cloud consulting services, we created a virtual machine for their interviews where the DevOps candidates can configure docker containers and use Terraform to instantiate them. Yes, we take very seriously the opportunity to recreate the work, the skills-based work that one would do on the first day of the job and essentially ship them their laptop that they will use on day one.
Mark:
There’s a lot of talk out there that it’s hard to find talent, especially technology talent, and here’s a pretty sophisticated tool. Are those things in conflict that you have employers really needing to find people, but then turning to technology to dive in deeper and make it possibly a little more challenging for the candidate?
Dan:
Well, there’s no doubt that the supply and demand of engineers is such that there’s not enough engineers to meet the demand. Every company essentially is, in my professional background, has become a technology company. Every company has moved its operations to the cloud. Every company needs front-end engineers, back-end engineers, [inaudible 00:08:06] engineers, data scientists, now DevOps engineers, cybersecurity is an emerging field, but virtually every company needs to build their skilled base in. It is true that the supply isn’t what is needed. That said, especially pre-pandemic, we were pretty anchored to our swanky offices with toys and food and beautiful views, in order to attract top talent and we would recruit within a commutable radius of different locations. That has gone out the window with the pandemic and the entire globe has become a company’s talent pool. As a result, your criteria for hiring people has become more skills-based.
I want to verify that they can do what we need them to do, as opposed to other more subjective criteria about how will they fit into our office and our office culture. And the plain reality is, interviewing someone who may be on the other side of the globe in a different time zone, requires you to verify that they are who they are and that they have the skills that you need on the first day on the job. I would also say that it’s no secret now that the economy is going through some adjustments post pandemic, and the number of candidates applying for jobs is actually rising. And as a result, companies have streamlined their recruiting departments, and this typically happens in every recession I’ve seen in the recruiting industry, the first people to go or the recruiters, the hiring managers still expected to fill the positions when someone leaves a job.
And in the case of most companies, the engineering hiring marches forward, but now they’re recruiting across the globe. What Filtered does is offer an opportunity for a company to automate at scale, a much richer, deeper interview. The final thing I want to say is that if you are a candidate and you’re a high-end engineer, it did matter to go visit the office that they were going to require you to drive to every day. You did want to see where you’re going to work. But now, the only way to really understand what it’s going to be like to work at a new company, where you may never visit someone in person or very rarely, is to understand what the work’s going to be and what their expectations are of you. And so we’ve had over 50,000 candidates rate us and give us an over 4.6 rating in terms of experience. We believe that providing a job simulation provides the candidate the information they need to decide the jobs a fit for them.
Mark:
What is the difference between say, coding tests and a job simulation?
Dan:
Well, a coding test isn’t literally doing the work. It is you’re asking a question, typically there’s multiple choice answers and you can pick one. There’s not a great opportunity to show your work, so you may guess between two and get it wrong, but most importantly, it’s not literally what you do on the job. You don’t take tests on the job, you solve problems. And it is a judgment call on what kind of person do you want to hire and what kind of problem solver do you want to hire? And so I really like what that CTO said when he said that the best way to interview a software engineer is to watch them code for an hour. Some hiring managers may want speed to be an important factor. Some may not care about speed and only care about how creative they are at solving a particular problem.
What we try to do is work with every client on what is your tech stack? What are the tools that you’re going to have your candidates if hired use on the first day of the job? Why don’t you provide some of your code. A lot of, for example, first things that new engineers do on the job is debug code. Why? Because they learned the code base that they’re going to be working with over the next few years. We’ve created simulations to debug code and it’s testing a real live skill debugging code that you didn’t write or refactoring code you didn’t write. It’s assessing that skill. That’s not a test, that’s a real simulation of what you’d be doing on the job.
Mark:
How do candidates feel about all this?
Dan:
I do believe candidates are tired of lots of things. We hear all the time that candidates don’t like a lengthy process that goes on too long. Filtered allows companies to make a quicker decision, to make decisions faster and the candidates will like that. Candidates don’t like taking coding tests that don’t teach them anything about what the job is going to be like. As I said, we provide candidates a real first day on the job work experience. Candidates don’t like being asked questions that they see as beneath them, that didn’t detect that they already have the certain skill and they don’t obviously being asked questions that are not a fit with their skills.
We believe in job transparency, in providing a candidate as clear a picture of what the job would be like so they can self-select in or out. But I think candidates who worry about leaving one job for a new job, having never physically visited with any of the hiring managers they’ll be working with, really do appreciate the opportunity to prove their worth, show their skills, demonstrate as opposed to talk about, their skills and allow them the opportunity to make the decision based on what that experience was like.
Mark:
Now, years ago when I was in school for journalism, we were told not to agree to an editing test if we weren’t allowed to use a dictionary. And the thought was basically that real newsrooms in the real world have dictionaries, so it was an unfair requirement. Does that kind of situation take place or is it introduced by some employers where the assessment or the test or the experience, makes a task more difficult than it really needs to be?
Dan:
That’s a really good question. Actually, it makes me realize, I should have also said that, for example, in our fraud detection capabilities, we don’t limit where a candidate can go to get help on fulfilling the job simulation challenge. We just track it. And so, the hiring manager can watch a candidate go to their own GitHub depository, pull code in and show that they particularly know how to do the kind of work that you’re going to ask them to do. Or they can just do a Google search and look for ways to solve the problem in a way that the company may not like because it reveals that they don’t know as much as they should know about a particular language or area. It’s up to, in other words, the hiring manager to decide.
But our belief is, that what you just said about being a journalist back in the day where you’re going to have a dictionary several, and over time online dictionaries that you can use that, that is the way you should simulate a job. They’re going to be working typically from home in their home office. They’re going to have lots of resources at their fingertips. You’re hiring them to navigate that world on your behalf to solve problems. That’s what you want to verify in an interview. We try to recreate exactly what it will be like when they’re working at home from their home laptop.
Mark:
This technology, does it expand the employers reach in terms of finding candidates and vetting them globally as opposed to locally? Does it change the amount of homework that they have to do?
Dan:
Well, it certainly enables companies to be much more comfortable interviewing people anywhere in the globe, because it is a skill-based hiring process, less subjective. It verifies that people can do what you need them to do, regardless of where they’re located. I think companies still limit their talent pool selection to the regions they know, the schools they know, the companies they know, because they’ve been looking at these resumes for years and years, and in order to save time, limit their talent pool to certain schools, certain companies, certain words on a resume, certainly they use their ATSs to help select candidates who put certain words in their resume. We believe that that’s an inexact methodology, I’m not criticizing, it’s just not field-based. And so we do think that this enables companies to broaden their funnel and be comfortable taking in candidates from anywhere in the globe. Your second question again was?
Mark:
Don’t employers still have a lot of research to do? This is giving them a more detailed and probably more accurate review of the candidates working, but it doesn’t erase a whole lot of responsibility for the employer, does it?
Dan:
No, it doesn’t. But that’s why we provide the hiring managers and the recruiting professionals, not only a great library of both environment images as well as challenges, but we work with them to create challenges that they may, as I said, with one company wanting to assess the ability to use their own robotic operating system. It does, I think require, hiring managers and recruiters to sit down and ask the question, we will be happy or unhappy after 90 days if the candidate has accomplished what, with what tools? What are we going to have them do on the first day of the job? And that then the articulation of that, we recreate it on that virtual laptop and then they can feel much better, that the candidates they actually interview in person have been embedded to do the job.
Mark:
Dan, it’s been great talking with you. I really appreciate your time and I hope you’ll come back.
Dan:
I would love to and I very much appreciate your time as well.
Mark:
My guest today has been Dan Finnigan, the CEO of Filtered AI, and this has been People Tech, the podcast of the HCM Technology Report. We’re a publication of Recruiting Daily. We’re also a part of Evergreen Podcasts. To see all of their programs, visit www.evergreenpodcasts.com. And to keep up with HR Technology, visit the HCM Technology Report every day. We’re the most trusted source of news in the HR tech industry. Find us at www.htmtechnologyreport.com. I’m Mark Feffer.
Image: Wikimedia Commons
