How to effectively manage brilliant yet difficult big data superstars

Hiring a talented data scientist is the goal. But when that talent gets in the way of getting work done, you have a problem.

Image: fizkes, Getty Images/iStockphoto

I recently saw an ad for a data scientist posted on Glassdoor. The ad  said, “No prima donnas, no whiners.” The term prima donna means “a vain or undisciplined person who finds it difficult to work under direction or as part of a team.” We’ve all worked with them, and some of us have even managed them.

SEE: Cheat sheet: Data management (free PDF) (TechRepublic)

Dealing with difficult people is one of the most challenging aspects of management; however, it can be a constant reality for companies deploying data science teams that are stacked with big data and algorithm superstars who know they are good at their jobs. You quickly recognize that while your goal was getting a good data science team with the requisite expertise in place, you also need to ensure everyone stays on task and works well with others.

This isn’t always as easy as it seems with top talent. 

Lesson learned by not getting a top talent on my project

Years ago, I was managing a project that required an extremely high level of system programming expertise. There was a data scientist on staff who had the expertise, but other project managers wanted her services, too. Repeatedly, I tried to get her engaged in my project, but she was always busy. Finally, fearing my project deadline would pass, I decided to go with a more junior person who lacked the expertise but was enthusiastic. The project was successful, and I’ve never regretted the move.

What I learned from the experience was that an extremely gifted person in a highly sought IT discipline can be so revered, sought after, and impressed by his or her own knowledge that they believe they can dictate the terms of work and projects, and often they can when managers are too dependent on the individual’s work to risk a confrontation.

Unfortunately, tolerating difficult behavior places a company at an even greater risk because it is consenting to being held hostage by people who only want to work on their own terms. This can damage the morale of your entire data science team and adversely affect projects.

No manager can afford this, so if you’re managing big data or data science projects, here are several recommendations for working with brilliant but difficult people.

Compliment but never patronize

Technology gurus in big data and other IT disciplines have devoted years into being the best at what they do. They are understandably proud of their accomplishments.Managers can respect and acknowledge these top-level technical talents in project strategy meetings and by encouraging them to take technical leadership roles on projects and to mentor others.

What you don’t want to do is just patronize the person by lavishing him or her with praise that you hope will meet their ego needs. Everyone, whether a junior staff member or a technical guru, can spot insincerity.

Treat everyone the same

When staff sees that you’re interacting with everyone and praising and critiquing work irrespective of who is doing it, your team and project morale will be healthy. You can manage by walking around and interacting widely with your staff. Your heavy hitters will observe this and see that everyone on the team is valued, and that there is no one so special that the world should stop.

SEE: 10 skills employers need in a data scientist (TechRepublic) 

Talk to the superstar about problematic behavior

There are managers I know who are afraid to approach their top technical performers with criticism because they are afraid they are going to alienate or lose them–this is a mistake. If you have a brilliant technical performer who is ripping apart your project and your team, it’s time to have a talk. Before doing so, you should touch base with your superiors to explain the situation and ensure that they are behind you. You should also have a consultant or a junior person in the wings to take over if the individual you have to talk with decides to leave.

Just because someone is talented doesn’t mean they run the show. Good management can help keep the entire team happy and working together well.

Also see