How to land a Data Engineer job in the US ?
$100k+ salaries ! Personal story included ! MS in USA students must read !
Am I worth writing this ?
I was a Data Engineer with Expedia Travel Group for 4 years and faced 10+ interviews for Data engineering with multiple companies. I did my Master’s in MIS and cracked data engineer roles with Expedia and Disney.
Let’s begin now..
What does a data engineer do ?
Bring data from point A to point B.
Bring data from point A to point B. That’s all what a Data Engineer does. And there could be some modifications that could be done along the way while bringing data from Point A to point B.
Point A can be different types of databases. Point B can be different types of databases. And data can be modified or transformed between point A and B.
So a Data engineer will write code (in python or sql or similar languages) to transform and transport data between various data sources and destinations.
Certifications required
These certifications are required to improve your interview chances. But not mandatory.
If you dont have any work experience, Completing 1–2 of these will definitely improve the chances of your resume being picked by the recruiters.
- oracle sql certification (oracle certificate adds an authority to your resume)
- python certification by Udemy or your university course
- big data certification by Udemy or cloudera or your university course
- data warehousing certification
Types of data engineer roles and Interviews
Role-1. Software Engineer oriented Data Engineer:
These are roles where the candidate is required to code deeply, though the roles are Data engineering. The candidate is expected to know Data structures. There ends the matter ! You may be called to the interview because you have Hadoop etc on your resume. But you will be judged on your DS skills too !
Role-2. SQL & Data warehouse oriented Data Engineer:
These roles are where Data engineers work more on data warehousing problems. Data warehousing problems such as:
- building SQL cubes or working on MS SQL
- or working on Pentaho/SSIS/Talend/Redshift
- and more proficiency in such technologies
Your interviews will test you on use-cases of those technologies — given a problem, how did you solve that problem using one of those relevant technologies.
Also some data modeling problems.
Role-3. SQL & Big Data oriented Data Engineer
These roles are where the candidate will be tested on small Python programs, Hadoop knowledge, data modeling problems, in-depth SQL proficiency.
The SQL proficiency can also be tool specific. For example: Oracle will have some functionalities in-built such as lead, lag etc. The candidate will be questioned on them as well.
Role-4. ML oriented oriented Data Engineer
These data engineers will not be tested on ML. However, your previous work experience or background about ML can be asked in the interview.
If you’re good with SQL, Python and Hadoop, and you have a good ML or Data mining background with the university projects or even theoretical concepts , you will have a higher weightage with your application.
Interview success vs rejection
We explored 4 different types of data engineers above. Given your background, interest and experience specified in your resume, you will get picked for one or the other roles.
Sometime you’re good with Data warehouse (Role-2), but might be picked for Big Data Engineer (Role-3) interviews. And there could be other candidates in the interview loop who are good with Role-3. And they could be better than you. Eventually you might get rejected even though you have done a good job in the interviews.
That is completely ok.
Understand that you’re good with Role-2 and Role-3 requires a different level of skillset which you can eventually possess, but there are better candidates for that role as of now. Those candidates might have more experience in Role-3 than you have.
Conclusion is: in such cases, where there are rejections in DE roles, understand that you’re good with your knowledge, it’s just that the roles didn’t align at this moment.
Personal story alert
I have personally seen such a thing happen to my friends. Both of my friends attended an Amazon interview for the same role. Both of them were roommates. Both are really talented.
One is very good with SQL and purely a SQL focused person with previous experience in Databases.
The other friend is excellent with ML, did all his certifications with ML, and he is equally good in SQL, he also has a previous experience but not database related. But he can do the job if given.
Both of them did Oracle SQL certifications.
The first friend who was more SQL focused got the positive result in the interview as Amazon DEs are more SQL focused. While the other friend eventually got into an excellent DE role that’s more ML oriented in a Fintech company.
Both got into the roles of their choice, but in different companies.
So rejections are for a reason, rejections wont always measure your skills, they measure your suitability to the role as well.
Pay scales?
All the 4 roles of Data Engineers are very well paid. Let’s take Seattle as the job location for example.
All of the roles will be paid around $100000+ as the base salary. But note that Role-1 (software engineer/DE) will definitely be paid higher than the other 3 roles for sure. If you clear Data structures, you will be paid more for sure.
The difference will be $10k approximately per annum. Role-1 engineers will be paid around $110k whereas others might be paid $100k.
These are just rough numbers to give you an idea.
Future career path?
Data engineer roles are very matured roles now. Future career prospects look very bright for DEs.
Some of the future roles for DEs:
- Sr. Data Engineer
- Sr. Software engineer or software engineer-2
- Product manager or Product owner, Data
- Data lead/architect
- Manager, Data engineering or Director, data engineering
