Software Engineer, Data Job Description
Software Engineer, Data Duties & Responsibilities
To write an effective software engineer, data job description, begin by listing detailed duties, responsibilities and expectations. We have included software engineer, data job description templates that you can modify and use.
Sample responsibilities for this position include:
Software Engineer, Data Qualifications
Qualifications for a job description may include education, certification, and experience.
Licensing or Certifications for Software Engineer, Data
List any licenses or certifications required by the position: AWS, SSL, MCSE, RHCE, ITIL, SSH, MDG, SAP, GCP, RDS
Education for Software Engineer, Data
Typically a job would require a certain level of education.
Employers hiring for the software engineer, data job most commonly would prefer for their future employee to have a relevant degree such as Bachelor's and Master's Degree in Computer Science, Engineering, Computer Engineering, Information Systems, Science, Technical, Mathematics, Education, Statistics, Software Engineering
Skills for Software Engineer, Data
Desired skills for software engineer, data include:
Desired experience for software engineer, data includes:
Software Engineer, Data Examples
Software Engineer, Data Job Description
- You will be a team player, yet take respobility and own delivery of modules, fucntions of project, product
- Be a key member of the data platform development team, which is composed of local and outsourced development teams in different time zones
- Perform research and development to evaluate new technologies, ideas and communicate value for company
- Ability to work with business users, product managers, QA, Release, and UI and Platform Engineering teams
- Must have complete understanding of agile methodologies
- Amass vast quantities of information that can inform investment decisions
- Provide that information to our cutting-edge trading and analysis platform
- Collect, parse, and clean terabytes of data used for research and real-time automated decision-making
- Excellent Training and Mentorship skills
- Experience in authentication, authorization & related security protocols
- 3+ years of professional experience working with Hadoop stack (HDFS, MapReduce required
- 3+ years of experience or demonstrated fluency with Java or some other object-oriented language
- Cloud Computing experience (AWS, Azure)
- Experience with logging and alerting platforms is a plus
- Hands-on high-performance development experience on big data
- Experience with big data (Spark/ Hadoop)
Software Engineer, Data Job Description
- Understand end to end product and deliver on the module you are working
- Design and build reliable, easy to use data pipelines and data systems
- Triage, identify, and fix scaling challenges
- Perform cost-benefit analyses of short-term needs vs long-term data scaling and company growth
- Educate product managers, analysts, and other engineers about how best to use our systems to answer hard business questions and make better decisions using data
- Help establish new team and define the culture of how you work
- Work with colleagues around the world on integrating critical data
- Execute on the product backlog – implement address validation service, improve account matching automation and much more
- Influence how the team runs
- Design and build big data collection systems
- Experience in implementing RDDs, Map Reduce jobs
- Understanding of online advertising landscape and technologies is desired
- Experience with source control management systems, such as Git
- Knowledge of Design Patterns, and asynchronous systems, loosely coupled systems
- Cloud service deployment technology (AWS, Chef, Puppet)
- Test-Driven Development, Unit Test, Pair Programming
Software Engineer, Data Job Description
- Design and improve real-time data ingestion infrastructure
- Design and develop data pipelines and streaming systems that are correct, scalable and fast
- Help customers design and implement data schemas and product logging
- Work with engineering organization to define data ingestion standards, work with your customers to develop suitable data solutions
- Develop and test data engineering solutions including data
- Design, develop, test, deploy, maintain and improve large-scale distributed systems
- Analyze and improve the efficiency, scalability and stability of distributed systems
- Perform analytics on large-scale distributed systems
- Design and build components that seamlessly run on-premises and on cloud platforms
- Work collaboratively with other team members within an agile software development process
- Ability to make extensive use of proprietary in-house file systems, databases, and data flow control systems built in Java and C++, with new languages and technologies continuously being evaluated
- Architecture chops
- Bachelor’s degree or better (or equivocal experience) in software engineering, information systems, or computer sciences is required
- Expert in applicable object-oriented software languages (preferably Java, C#, C++)
- Experience with multi-threaded and multi-core development strategies
- Advanced developmental skills using Object-Relational Mapping (ORM), Language Integrated Query (LINQ) or other development-data relational modeling (NHibernate)
Software Engineer, Data Job Description
- Design and implement production Big Data environments using modern technologies
- Recommend and implement creative solutions for improving response times for large data sets
- Collaborate openly with stakeholders and clients to continuously improve the product and increase adoption
- Support CCDS infrastructure including data interfaces, processing pipelines and general programming tasks
- Analyze and resolve customer and technical problems in developed/implemented systems
- Analyze results and implement changes to improve performance, processing and utilization of computing resources
- Develop and maintain system documentation training and reference materials for the community using wiki and knowledge management tools
- Knowledgeable in the concepts associated with MDM services, data model, technical features
- Work with the team on modelling complex relationships in heterogeneous data environments
- Build, validate and optimize large-scale, corporate-wide analytics tools
- Master's or bachelor's degree in computer science or another technology-related field
- Demonstrated experience participating in open source communities
- BS or MS in Data Engineering, Statistics, Computer Science or engineering field or equivalent experience/knowledge
- Solid Background or rich experience around Data Engineering, Data Science, Data Mining & Analytics and/or Statistics
- Knowledge of PowerBI, SQL, including database design and query optimization
- 7+ years designing and developing software with a proven track record on products that ship worldwide
Software Engineer, Data Job Description
- You have a rock-solid foundation in Computer Science (data structures, algorithms, software design)
- Design and improve a generic ETL framework, that is used by hundreds of users in the company
- Develop a Data Portal that is designed as the central data encyclopedia for all users in the company
- Identify new tool ideas that multiply the impacts users make, using data
- Be the technical customer evangelist who understands the needs and the desire from customers and can translate into the engineering development and optimization ideas
- Work with engineering organization to define data standards, work with your customers to develop suitable data solutions
- Work directly with both internal entities accounts and sales managers, R&D, along with clients
- Understand how to build solution from scratch and familiar with different software architectures and frameworks - MUST
- BS in Software Engineering or related field
- A passion for shipping quality high-performance code
- Experience running and supporting big data implementations in a production environment
- Deep expertise in building structured and unstructured database systems
- Strong interpersonal communication skills to work well with others
- Ability to communicate technical information in a clear, precise, and actionable manner to a broad ranges of audiences
- Expertise with Dataflow, Flink, Spark, Storm, Kafka
- Familiarity of industry best practices — code coverage