While the future is uncertain for many types of jobs, folks in the tech business can rest easy knowing there is substantial demand for their services, particularly for data engineers, which was the focus of a new study from Burtch Works.
The Burtch Works salary study finds that hiring for data engineers is set to take off from now through December. According to its survey of 320 data engineering professionals, which took place from summer 2020 to summer 2021, 81% of companies are planning to hire additional data engineers in the second half of 2021.
“This is a significant increase over our survey for the first half of the year,” the company writes in its new report. “…[T]he market is booming.”
While there was some uncertainty over data jobs in early 2020 due to COVID-19, hiring managers have a newfound clarity when it comes to stockpiling data engineer talent, Burtch Works says. Companies are scrambling to not only expand existing teams, but they’re trying to create new teams from scratch, which is “exacerbating the supply/demand issue,” it says.
The Linux Foundation’s research arm came to a similar conclusion in its recent report on the state of open source employment, but with some caveats. While its recent survey found that 50% of hiring managers plan to increase their hiring of open source professionals in the next six months, that was down from 2020, when 56% said they were ramping up hiring.
“Where 2020 brought hiring urgency and a need to ramp up open source talent, 2021 looks to be the year of cautious rebalancing,” Linux Foundation Research stated in its report. “We believe the 2020 spike in demand was due to last year’s survey being conducted toward the end of the first wave of COVID-19 lockdowns…”
But other signs point to robust demand for professionals with open source technology skills, including a paltry 6% of hiring managers saying they plan to hire fewer employees, according to the survey. Also, solid organic growth in companies was reported by 50% of the folks who took the survey, while 38% report having talent shortages in open source tech.
“These are both consistent with a wider economic rebound, as organizations are again experiencing growth and continue to invest in digital transformation activities, which drives further open source adoption,” Linux Foundation Research states in its survey.
Data Engineering Salaries
Burtch Works is best known for its salary surveys, and it did collect this data from data engineers. However, since this was the first time surveying data engineers specifically, it didn’t have earlier figures to compare to.
The average salaries for entry-level (five years of experience or less ) data engineers was $106,000, according to the survey, moving up to $150,000 for more experienced data engineers, which the company calls individual contributors. The average salary for a project manager (including leads, senior managers, or associate directors) was $170,000, while managers with more strategic jobs (i.e. vice president, heads of engineering, senior directors) was $228,000.
“While we do not have a salary report examining data engineers for previous years, we anticipate that salaries are currently on an upward trend for these professionals,” the company says. “Data engineers are in high demand as many companies continue to invest in digital transformation efforts to survive in today’s data-driven economy.”
Bonuses also factor into the compensation puzzle for these hard-to-find professionals, Burtch Works says, with individual contributors getting from $22,000 to about $25,000, and managers clocking in with a $32,000 to $64,000 range. Two-thirds of data engineers had advanced degrees (62% master’s and 5% Phds), according to the study. Many of these degrees are in computer science and information systems, while some had degrees in analytics, natural sciences, and social sciences. Only 13% of data engineers are women, versus 87% who are men.
Skills In Demand
Burtch Works spotted an increase in demand for engineers with cloud experience. “Among both data engineers and their employers, we’re seeing a strong preference for AWS, Azure, and GCP, and an increasingly definitive move away from Hadoop in many cases,” the company writes. GCP, in particular, is becoming “a much more prominent player than in previous years.”
The list of technical skills that are common among data engineers probably won’t surprise you: Python, Scala, Java, relational and NoSQL databases, Hadoop, Spark, and BI tools are in the running. Building data pipelines with ETL and ELT tools, and designing and supporting data lakes and data marts, remain core to the job description. But Burtch Works also noted an increase in demand for Kubernetes and Snowflake as
Kubernetes is “becoming more widely used as there has been an increase in data engineering roles requiring a focus on agile development process, continuous integration/continuous delivery practices, and a DevOps proficiency,” Burtch Works says.
That data point was backed up by the Linux Foundation Research, which found cloud and containers were the hottest skills, followed by DevOps, Linux, and networking.
This article originally appeared in Datanami.