The morning briefing at Algorithm, Inc. always began the same way: Dr. Chen presenting the latest efficiency metrics while the junior analysts tried to look attentive. Mike had been here six months, and he still couldn't shake the feeling that something was fundamentally wrong with their work.
"Today's focus is the loan approval algorithm," Dr. Chen announced, his voice smooth and confident. "We've achieved a 34% improvement in processing time, and accuracy is holding steady at 94%. Excellent work, team."
Mike raised his hand. "Dr. Chen, I have a question about the accuracy metrics."
The room went quiet. Questions were not encouraged at these briefings.
"Go ahead, Mike."
"I've been reviewing the rejection data, and I noticed something concerning. The algorithm seems to be rejecting applications from certain neighborhoods at a much higher rate than others, even when the financial profiles are identical."
Dr. Chen's smile didn't waver. "That's not a bug, Mike. It's a feature. The algorithm has identified patterns that correlate with loan default risk. Neighborhoods with higher default rates naturally receive more rejections."
"But what if those patterns reflect historical discrimination rather than actual risk? What if we're perpetuating the very biases we claim to eliminate?"
The silence that followed was deafening. Dr. Chen's eyes narrowed slightly.
"I appreciate your concern, Mike. But you need to trust the data. The algorithm doesn't have feelings or prejudices. It simply processes information more efficiently than any human could. That's not bias—it's progress."
After the meeting, Lisa caught up with Mike in the hallway. She was one of the few colleagues who didn't treat him like a pariah for asking questions.
"You need to be careful," she whispered. "Chen doesn't like people questioning his algorithms. Especially not in front of the whole team."
"I'm not trying to cause trouble," Mike replied. "I just want to understand. If our algorithm is discriminating against people, shouldn't we fix it?"
Lisa glanced around, then pulled him into an empty conference room.
"Look, I've been here three years. I've seen what happens to people who ask too many questions. They get reassigned to 'special projects' that never seem to materialize. Or they get let go for 'performance reasons.' The company values efficiency above everything else—including ethics."
"That's exactly my point. What good is efficiency if we're hurting people?"
Lisa sighed. "I'm not saying you're wrong. I'm saying you need to be smart about this. Document everything. Build a case. And when you're ready to act, make sure you have allies."
Mike nodded slowly. She was right. He couldn't fight this alone. But he couldn't stay silent either. Not when he knew what the algorithm was doing.
Two weeks passed, and Mike heard nothing about his report. He tried to focus on his regular work—optimizing ad placement algorithms, fine-tuning recommendation engines—but his mind kept drifting back to those loan applications.
Lisa found him in the break room, staring at the coffee machine like it held the secrets of the universe.
"Still thinking about the loan algorithm?" she asked.
"I can't help it. I keep seeing those rejection letters. Real people with real dreams being told no by a machine that doesn't even know them."
Lisa poured herself a cup of coffee. "Have you considered that maybe the algorithm is right? Maybe those neighborhoods do have higher default rates, and the algorithm is just protecting the bank's interests."
"That's exactly the problem. The algorithm is protecting the bank's interests, not the people's. And the reason those neighborhoods have higher default rates is because banks have been denying them loans for decades. It's a self-fulfilling prophecy."
Lisa considered this. "So what are you going to do?"
"I'm going to run my own analysis. Compare the algorithm's decisions with actual outcomes, not just predictions. See if there's a gap between what the algorithm thinks will happen and what actually happens."
"That sounds like a lot of work."
"It is. But if I'm right, it could change everything."
Mike spent the next month working late into the night, gathering data from public records, cross-referencing loan applications with actual repayment rates. The results were worse than he had imagined.
The algorithm wasn't just reflecting historical bias—it was amplifying it. Qualified applicants from certain neighborhoods were being rejected at rates 40% higher than equally qualified applicants from other areas. And when those rejected applicants managed to get loans elsewhere, they repaid them at the same rate as everyone else.
The algorithm wasn't predicting risk. It was creating it.
Mike compiled his findings into a report and submitted it through the official channels. He expected pushback, but he wasn't prepared for what came next.
The subject line made his stomach sink. This wasn't a discussion. It was a defense.