Is Data Engineering less stressful compared to software engineering
Hi Guys,
I'm an Java SDE and I have had bad luck twice now, I got stuck in a company with a project which was super complicated and many people left from there, another company I joined was a service based company where I got stuck in a legacy java project and management having no experience in tech and creates pressure to deliver quickly.
I am sick of this for the last 2 years now, I want to change my field, is data engineering less stressful compared to software in terms of work complexity.
And what are the most used technologies that companies use for data engineering, like for backend development, spring boot java and node.js are mostly seen. For data engineering what are those technologies.
One interview, 1000+ job opportunities
Take a 10-min AI interview to qualify for numerous real jobs auto-matched to your profile 🔑
In DE, some projects would require you to have lot of domain knowledge(specific to the client's operations). Until you acquire that. Those kind of projects will be stressful.
Other than that most of the times it will be repeative work. If not it will be kind of medium level leet code problems, like if n inputs from m no:of tables needs to be transformed into k outputs of L no:of tables. Somethings the i/p to o/p transformation can get really complex.
If a project have bad architecture i.e, the data fragmentation is too much. Instead of 20 to 30 jobs across 2 or 3 layers/stages. It is split across 50 to 100+ jobs across 4 to 6 stages.
Those kinds of projects will be really stressful.
So in other words there's no guarantee that the project will be less stressful, I'll try to avoid any small company in that case and especially an Indian small service based company, licking clients a** and putting all pressure on developers are their favourite hobbies, it's a meat grinder that's for sure.
De is boring work for setting up the same pipelines,no innovation nothing
Don't really care, I just want less stressful work, don't care if it is repetitive, as long as it's easy compared to software. Do you know what tech stack should I focus on for data engineering if you are aware
It's mostly like a sanitation worker. You won't know what shit you have to clean up on a given day. All fancy tools and frameworks but imo, it's like the administration department playing around with their excel sheets.
Okay, do we have same agile methodology in Data engineering as well, like in software we have to give estimates and then deliver the work under those, if we can't then manager eats out brain. Do we have similar situation in data engineering also, I just wanted to understand if it is easy compared to software engineering
Not sure whether DE is less stressful or more but Your professional stress mostly depends on the project and team you are assigned to.