147: The Surprising TRUTH About Data Science Careers (ex-Amazon data scientist Daliana Liu)

I think a lot of times we think it's important to constantly grow your technical skills, but that only get you somewhere. So basically, if you imagine the career trajectory from junior data scientist to senior data scientist and later staff and principal scientist, you'll see the requirement for technical skills slowly. The increase is there. Not that high. And however, it requires more communication skills, leadership skills, influencing skills, higher level you want to, um, become.

So Dalyana, you have almost 300, 000 followers on LinkedIn. You're a LinkedIn top voice. You're the host of the data scientist show. Uh, you've worked as a data scientist, uh, at Amazon, and now you're kind of doing your own thing, teaching other people how to be data scientists. Thank you so much for coming on the podcast and, uh, I'm so excited to have you here.

Yeah, thanks for having me Avery. It's been a long time coming.

Yeah, I've been on your show and now you're coming on, on mine, but I'm really excited for, for me to get to know your story better and also for our audience to know your story more. And also just know more about like what it actually takes to be a data scientist, you know, specifically at a company like Amazon. So for those who maybe haven't followed you in the past, can you just give like a quick overview of what your career has been like?

Yeah, so I started studied applied math in college when I lived in China, and then I felt it was too much theories, and I wanted to learn something more practical. So that's when I started to get into statistics. Got my master in, um, University of Irvine, University of California. Uh, and then I got my job as a business intelligence data, data analyst. So at that time the word data scientist wasn't invented, but I was basically doing, uh, data science generalist work. Uh, I'm doing. Analysis for the marketing team, build time series model. And then I, uh, got into Amazon. I moved from LA to Seattle. Uh, my title was again, business intelligence engineer slash statistician. I think that's a. Basically perfect kind of role for a data scientist. Uh, and later I work on experimentation, AB testing, product analytics. So that's the first few years in Amazon. And then I got into machine learning and deep learning and moved to, um, Amazon web services and also grow to a senior data scientist.

Very cool. And I think it's so interesting that you started your career as a data analyst or like with that data analyst title. Do you recommend that for others? Like starting as a data analyst or was that role already kind of a data scientist role, but just had the title of data.

I think today, if you look at what people do under a data scientist or data analyst role, it's so different. For example, at Facebook, there are product data scientists. Don't do much machine learning and modeling. And they write a lot of SQL and they probably also use Python and in some other companies, there might be a data analyst also doing some modeling. So I would say it really depends on the company and what a role specifies, but in general, data scientists do. Use Python more, do a little bit more automation compared to data analysts. And some data analysts, they work more as a business analyst. They work closely, very closely with, um, stakeholders. I don't think there's a good or bad to start your career where it really depends on what you're interested in, what kind of job, uh, market look like, and, uh, uh, Regardless of where you get started, you can grow and become either a manager or a principal data scientist and analyst in your, uh, in your career track.

I agree that there's probably not a right or a wrong necessarily, but one thing, one thing that is really interesting is just that those titles kind of being all over the place. And I think that's true today. I mean, I think it's probably more true back then. Then, but even today, like I see some of the strangest titles, like I've seen a data science analyst before, uh, or data analytic scientists. Yeah. And I'm like, I don't know exactly what, what those roles are. So if I had to ask you, what is a data scientist? What would you say the definition of a data scientist is?

Yeah. I think the data scientist is someone who uses data and some kind of framework. Could be experimentation, could be, uh, machine learning, or it could be some kind of statistics analysis to help their usually business, uh, stakeholder make better decisions. Um, and this decision could be one decision could be you automate a million decisions, making it into a machine learning model and eventually have some sort of business impact, meaning it help your company make more money. Uh, save more time, save money, et cetera. Getting more customers.

I like that definition. Do you, do you think that like, do you think that there's a difference? Cause like a data scientist is kind of what you just explained, right? And there's this whole field of study called data science. Do you think data scientists are the only ones that do data science or like, where do you see data science versus data analytics?

Yeah, I think now everybody does data science, not just data analysts, product managers, they have to know some data science, data science. They might not be the one that always writes SQL, but they need to understand it. And I've also seen a lot of automated analytics or machine learning tools. Maybe a product manager in the future can easily use those tools to create some analysis. Engineers. They need to know data science. In fact, a lot of AI engineers these days, they basically came from a software engineering background and then they learned machine learning statistics on the go. And of course, we'll talk about the overlap. I think the biggest difference is in general, I would say, See the data scientists, the people with a data scientist title, um, some of them work on machine learning. Some of them don't, but the ones who work on machine learning, deep learning have more engineering element in it. See them more often in a data scientist title. And for a data analyst. I think the data analysts today also do a lot of automation, but it probably lies in, uh, some of them might do some data engineering work or creating a dashboard, new automated dashboard, but it doesn't mean their work is easily automated. Need to communicate a lot with their stakeholders to find out what is the most important thing. What's the story you need to tell from their, uh, dashboard. Uh, so they probably use more SQL and some data analysts. I know they use a lot of Excel as well because their stakeholders are not technical.

I think that's like a good definition because really at the day. There is so much overlap. Um, and it really, like you said, depends on the company. So it, it's, it's quite difficult to, to actually draw a line on, uh, let's talk about some of the work that, uh, you've done in your career. So as a data scientist, you, you mentioned the term machine learning, which, which I think is a term that a lot of people hear, but maybe don't know the definition of. For, for me, it's basically just using some sort of, of math to accomplish some business problem. The biggest one probably is predicting, uh, what's going to happen in the future, but there's obviously other things like, you know, separating things into groups and stuff like that. Would you say that's kind of a fair definition of machine learning?

Yeah, I think so. It's basically learning machine learning is basically learning patterns from data. If you think about a pattern of let's just say you drink coffee on Monday, Wednesday, Friday, but don't drink coffee on the rest of the day. So if I have enough data, if you follow those patterns, say 90 90 percent of the time, I'm able to use a model to learn that. So I think that's the. Simplest machine learning probably just have like one parameters, which is the day of the week. Uh, and today, when we think about machine learning, it's more complicated. We're probably some models. If you think about the open AI had GPT probably have, uh, I don't know, millions, billions of those parameters.

Yeah, that that's pretty complicated stuff, but you've also worked on some pretty complicated stuff. I would imagine at Amazon, one of the things that I saw that you, you, you'd kind of co published with Amazon was essentially a soccer project, which I played soccer growing up. So I was a big fan. Yeah. And I think a lot of people listening really, really enjoy sports. Maybe they've seen the movie Moneyball or read the book Moneyball kind of highlighting the Oakland A's and how they used analytics to, to, you know, win a championship. Uh, can you talk a little bit about what you kind of did at Amazon with, with this soccer project?

I was at Amazon Web Services and, uh, our team at that time was called ML Solutions Lab. So basically we're a group of consultants. We help AWS customers, um, implement a machine learning, deep learning solution. So this customer came to us. They are a sports betting company. They have a lot of soccer game data and they want to see whether they can predict whether there will be a soccer goal in the next few seconds. for joining us. And so this is the first computer vision project I worked on. And it's a very complicated project because we need to analyze the videos. Uh, we basically, um, used a few different frameworks to chop the data, um, into kind of five second, five seconds, seven second clips, and then we have to manually Manually label the data into whether this moment is a goal, whether it's not a goal. And if you watch soccer, you know, sometimes a very intense, uh, how do you call it? Uh, attack. It looks very similar to a goal. So we also need to label that to train a model, to learn this is attack. This is not a goal. So a fun story is because the data came to us. We're not labeled. So me and my coworker spent two days. Just looking at those clips to label whether this is goal or it's not a goal. So over the two days, I think I probably watched hundreds of soccer goals. I don't want to watch soccer for the next couple of years. So that's the unexpected part of data science. Sometimes you need to do a lot of those type of data quality check, labeling. But you have to do those type of things because we, we label it in a very specific way that we know how to train a model. Eventually, uh, we used, uh, we experiment that on a few different, uh, video analysis, uh, modeling called, uh, um, I3D. So basically it's an inflated 2D. Using, uh, inflated 2D modeling to analyze, uh, the data and the way, how to simplify the business problem. Because like I mentioned, after tech, there could be a goal. It could be not a goal, maybe ended in like a corner. For example, we'll just simplify that into a binary problem. That's also, um, important way to tackle a very ambiguous, complicated problem. Sometime you might not, you might need to reduce the scope. Uh, for example, this, in this case, we reduce the problem space from a multi class classification problem into a binary classification problem. And then we train a model. When we came out with a classifier, uh, with a classifier, we Use a classifier to run through the entire game. Every five seconds, we run through that classifier and then see whether there will be a goal in the next few seconds. We also created a very fun demo. Basically, in real time, you can see a Uh, likelihood score of whether there will be a goal in the next few seconds. It can make the viewing experience more exciting.

Yeah. I saw, I saw the demo actually, and maybe I'll, I'll insert a little recording because it was pretty cool. Um, but what, what a cool project. Uh, and I think. Uh, there's so many different, different things. I think that listeners can, can learn from that one companies like Amazon. And honestly, a lot of companies are act as consulting companies a lot of the time. Uh, and so what, like a gambling company in this case, or any other sort of manufacturing company, or I don't know, whatever. Company that exists a lot of the times they like kind of outsource their analytics and data structure stuff to smarter companies like, like Amazon. And I think that's, that's good to know. And I think that's a cool role to sit in is basically you get to do analytics for, for multiple, multiple companies. I think that's really cool. And then the other thing I love that you said was, you know, I didn't seem like you were too much of a soccer fan necessarily, and you to become one. Uh, and that's sometimes what you have to do is like, you maybe don't have the domain experience, but. You can kind of need the domain experience when you're building machine learning models a lot of the time.

Yeah, exactly. And I also worked on a football, American football project when I was on the team. I knew, I mean, I, I know a little bit of soccer, of course, but I knew nothing about American football and I have to buy a book to read how football work, uh, to, to, you know, to our point, to understand the context.

It's crazy because yeah, it's just, there's the cool thing about analytics and data science and machine learning is it's really industry agnostic, which really means that you can take the principles, the machine learning, um, models and the machine learning algorithms and apply them to really so many different business problems. And so you could probably spend your whole life just learning about different industries and how to apply just one model to those, those different industries. Uh, which I, which I think is fascinating and one thing I want to give you credit for in your LinkedIn content. A lot of the times you're, you're obviously very technical. Like you use a lot of very fun buzzwords, uh, when you're kind of explaining that and you've obviously worked for Amazon. So you're obviously very technical, but one thing I really appreciate about your LinkedIn posts is, you know, sometimes they're technical, but other times they're like, Hey, you as a technical person actually kind of get more done when you focus on your non technical skills. Has that been true for you in your career?

Yeah, absolutely. Uh, I think a lot of times we think it's important to constantly grow your technical skills, but that only get you, um, somewhere. And after that, uh, I wish I could show you a plot. So basically if you imagine the career trajectory from junior data scientists to senior data scientists, and later staff and principal scientists, you'll see the requirement for technical skills. Slowly, the increases. Not that high, and however you require, it requires more communication skills, leadership skills, influencing skills, higher level you want to become. And I think once we get into the reality, there's no homework anymore, and there's no, uh, perfect data. And a lot of times the stakeholders are not even clear about what they want. And so it is essential to know. You know, from the beginning of the project, how to ask the right questions, how to work with the right people, how to find a project that actually have the high impact that can get you a promotion and later on, how do you influence the right stakeholders to get your solution in the right place?

It's a, it's a crazy concept because I think we, we like to think as technical people, the, the more technical you are, the more you'll get paid, the more desirable you'll be, the more influence you'll have at a company. And, and to be honest, it's just not true. Even if you're not junior levels, even when you're trying to get hired. It's not like the smartest person or the best person at SQL lands the job. There's often these soft skills, these people skills, these communication skills, uh, that come into play and, and really kind of make the difference between maybe a good data analyst, a good data scientist, and a great one. Um, one of the ones that you posted about recently, and I think you kind of just. Hinted at it just barely. And your answer was sometimes these stakeholders don't have a clue, uh, of what you want. And so one of the things you posted recently was like, one thing that can make you a great data scientist is getting feedback early. Can you expound on that?

Uh, when I started, uh, in Amazon, I wanted to show my manager where my stakeholders, my work only one is perfect. Otherwise I would feel embarrassed. But reality is. Sometimes you think you understand their request, but you don't. Or during the time when you're working on a project, their preference, their priority have changed. So it's important to constantly align with your stakeholders to make sure you understand their needs. And also, there's only It's very limiting what you can communicate three words, especially you're working on a data science project, whether you need to turn that into a dashboard or machine learning model. So you have to show them your demo. Um, I, in my career growth course, I always talk about show them a ugly demo first. Even if it's just in your, um, Jupyter notebook or in your, you know, SQL, uh, you know, editor, show them to let them know what's the, uh, what does the MVP look like, it's even better if you can create a very small UI so they can play with, they can get excited for, and when they see what. It might look like it gave them more idea. So it's not a bad thing when they tell you, Hey, this is not what I want. If you're only 20 percent of the project, but it will be a huge problem. If you're already at 80 percent of projects, actually you want to, uh, have those small tweaks and ask them, Hey, is this what you want? Or I have a few other ideas that I think that might help you. This is my proposals. What do you think? So have those conversations early can save you a lot of time. When you're towards the end of the project,

I'm sure, I'm sure you've seen this now as you've grown in your career. And I've seen it as I've grown in my career to the point now where I, you know, I code a little bit, but a lot of what I do is, is directing other people to code and stuff like that. And I realized that, you know, now I'm the stakeholder and I've become the, a bad stakeholder where I don't even know what I want half the time, what I'm asking, or when I do know what I want, I kind of stink at explaining what I want. Um, and so when people. You know, who, who are working under me, are able to come back with something quickly and be like, Hey, is this what you're asking? Uh, especially like in a meeting or in a demo, like a loom video, uh, because I like, like what you said, you, you can only say so much with words. It's almost like, like internet speed, right? That's basically like how fast information can transfer words is like, I don't know. 15 megabytes per second, but like an in person meeting, we're talking like gig speed. Um, there's just so much more communication, which is, which is, which is awesome. And that ultimately leads to what, what's called like adoption and people using your analysis. Um, and that's another thing that you've mentioned, uh, on LinkedIn that like. You can't really do data science just for, for funsies. You have to get it adopted. Can you talk a little bit more about that?

I think there was a data point a few years ago, probably over 80 percent of machine learning models fail. I think part of it is natural because there's a research or discovery nature in data science. Not everything has to be put in production. But a lot of times if What you have done become useless. Then from the company perspective, they wasted their time. You don't have direct impact. And from a personal growth perspective, if you don't have the impact, it's hard to define your contribution to the team, to our growth. And how do you advocate? Yourself for that promotion, when you build something, a lot of data scientists and also engineers, they want to just build something that they think is important or they think is cool. They just learn some model from a Coursera course. They want to implement that. I think that's a great way to learn. By doing, but when it comes to, um, doing work for a, most of the time for profit company, you need to think about is what I'm working on aligned with my team goal. I'm not helping my stakeholder or, um, this is the five goals. My manager tried to achieve. I'm not helping my manager. I might be a team player. It's better if you can align your passion to the impact. And sometimes the passion and impact might be a separate thing. There is one thing maybe you can exercise your passion for, for learning on your own time or take 20 percent of, you know, your, your work time, but make sure the 80 percent of your time, you're actually solving the useful business problem. And a lot of time, it could be a little bit, um. Boring and repetitive. Um, I think that's also an opportunity for you to create more impact, to see how can you, um, automate this? How can you also, sometimes you need to motivate yourself that again, aligning with the stakeholders, with the customer's pain point, stakeholder's request. Sometimes if you see how does that implement it, how it solve even just one person's problem, it can. Also make you feel more motivated to work on projects like that.

It's hard because in, in data, especially like in school, right? Let's just take like a normal, you know, college, maybe like a master's degree or maybe even an undergrad degree. Your master's was in what again? What did you say your master's was in?

In statistics.

Okay. And yeah, and my master's was in, was in data analytics technically. Right. Um, but like, I would imagine it was the same in your master's, but my master's was very theoretical. Um, and it was all about like, like, for instance, you, you might be interested in statistics of, of like getting a P value less than, you know, zero point or 0. 05. And you might be interested in like, okay. Like, can we make the P value lower? I know we can't really make P values lower, but like you might be interested in really low P value or, or in like my masters, it might be like, Hey, we're like 79 percent accurate, can I get to 81 percent accurate? So we're thinking in like P values and percentages. But really, like you said, most businesses are pro for profit. So they think in dollar signs and usually pretty much dollar signs only. Uh, so if we can't relate our analysis and our work that we've done into dollar signs, and it doesn't have to be dollar signs. It can be time saved. It could be lives saved. It could be. You know, people promoted, I don't know, whatever, whatever the, the key units, yeah, more users. That's, that's another good one. Whatever your team is focused on, you have to figure out how to get your analysis there. Otherwise you're not really helping the team out one and two. Like you said, your, your career growth is going to struggle because. Especially these bigger companies, like your promotions are kind of tied to the work you've done for the impact you've had for the business, basically.

Yeah.

Yeah. Okay. I, I agree with you there. I think that that makes a lot of sense. Another thing I think you, you mentioned in a LinkedIn post is like. Maybe you are trying to do that, right? But you're like, you're kind of struggling to, to like advocate for yourself. You're kind of struggling talk to your, to make, to make your work clear. Do you have any advice on like how to like make your, your work more known like in the company?

Um, yeah, so. Uh, you meant, uh, making, having more visibility in a company?

Yes.

Yeah. If you already work on a high impact project, you probably will work on sometime directors or VP. So I don't think you need to be kind of quote unquote famous in your company. Of course it helps, right? If you did deliver a high impact project and you give a talk. You have more visibility. Maybe there are other people come to, um, invite you to for collaboration. For example, when I published a blog post on the soccer project, which is talking about there are other teams reaching out to me, asking questions, looking for collaborations, but a lot of times you only need to be visible to a. In their circle of people, for example, the people who actually decide the road map of the team or the person who might be on the committee of your promotion review, I think a great way to do this is to. See if you can build a relationship with them to collaborate with them. And the first step is, uh, if you, again, don't know how to build a relationship with, you can go from a perspective to just learn from them, to get feedback and show them what's something you have been working on. Kind of similar to, we talk about getting stakeholder feedback. Um, if you want to bring more awareness for your project, for example, you're building a new tool that will improve. You aim to improve your team's productivity, maybe talk to the key users, potential users of this tool or some other stakeholders and show them a quick demo, um, ask them what's their pain point and get, get a user feedback. So when someone is. Involved when they, uh, give you ideas and you implement them, they feel they're part of the project. So later on, when they have some similar project, they're aware, Oh, there is someone I can talk to on that team. They're expert in this. So in a company, you don't have to be an expert. You don't need to work on one project for. 10 years and have a PhD in it to become expert. Sometimes if you deliver project end to end, you, you know, a lot of the domain knowledge and the business contact, you are an expert, let people, uh, by collecting feedback, um, talk to people one on one, um, sometimes help them. People know that you are the expert on this domain. And when you finish the project, share your work through an internal blog post, or you can schedule a lunch and learn session, et cetera. And, uh, I know we all have our own priorities. We're busy, but sometimes also need to, you can set aside some time to host. Office hours or, um, Q and a sessions, be generous with your time. Sometimes also goes a long way.

Very cool. And I think, I think that is awesome advice on, on increasing availability, uh, sharing your work. It's such a, seems, seems like you shouldn't have to do that because you're at work and it's like, why do I have to share this with anyone? Uh, but it can be such a big, uh, impact to your career. And, and others as well. Uh, well, Dalyana, this is the Data Career Podcast, and obviously you've shared a lot of good things about growing your data career. Uh, I want to ask you if you had to give someone who, you know, is listening to this episode, any sort of advice on advancing their career to the next level. What would you give them?

I have so many devices, very hard to come down to one. Yeah, I would say. There is, of course, it's important to understand how to create more impact for your company, uh, how to advocate for yourself. We are, um, in this kind of system, there's promotion, there's annual review. It's important to know how to play that game. Uh, but at the same time, it's also important to look inward. To know what do you enjoy, what is your goal, uh, what's your life goal beyond your, the, the next level of the promotion or the raise, I think is helpful for you to play the long game, um, when you know yourself better, so maybe. Uh, every quarter or every year said, uh, we're at the end of the year, maybe during the holiday season said, uh, one afternoon, just write down how do envision your life will look like. And then think about how could your career, your family, your friends play a part of it. So at the end of the day, the career is only one aspect of our life.

I think that's important to remember because it it's really easy to get lost in, uh, all in it all because. It's like, why do we work? We work to live. And sometimes it feels like we live to work. Um, so I think that is sage advice. Uh, Dalyana, thank you so much for coming on. We'll have all of Dalyana's, uh, links in the show notes down below. She's been working on something really cool as well. Dalyana, you want to talk about what you've been doing recently?

Yeah, so I'm working on, uh, more career coaching. So one course I recently launched is called the data science career accelerator. So we talked about how to, uh, improve your stakeholder management skills. How to be a great communicator. So all the soft skills we just talk about and how to build a relationship with our manager, how to create more impact and get a promotion you deserve. So basically we teach you all the required soft skills, leadership skills, communication skills that school didn't teach you. And this course requires you to be a data scientist for, you know, at least one year. Um, and, uh, a lot of, uh, the senior data scientists take this course too. They want to learn how to continue to expand their scope. So, um, I will share the link with, um, Avery.

Yep. We'll have the link in the show notes, uh, down below. We'll also have links to your social as well. So make sure you're following Dalyana already. Dalyana, thanks so much for being on the show.

Thanks Avery.