testsdfsd

Company Overview

Stream Systems (www.streamsystems.ca) is a leading-edge technology company that enables enterprises to optimize and make better decisions. Our SimOpti intelligence platform brings AI, machine learning and simulation to power business intelligence across complex operations, in any industry.

Position Summary

Reporting to the Software Development Manager, we are looking for a Data Developer experienced in continuous data process improvement across a range of data systems and subsystems. For this role, you will work closely with our data producers and consumers to enable Stream to automate data provisioning across data domains for multiple data use cases. Your data engineering expertise will implement data architectures, most importantly data mesh. Your ability to implement data technologies quickly and efficiently for any data layer is critical to our agile data flow.

Your deep understanding of the data value stream includes data ingestion, distributed data stores, data pipelines, transactional and analytical schema’s, self-service data as a product, federated data governance, data delivery through multiple data sharing API provisioning services (REST, GraphQL, Streaming, GRPC, etc.). You will implement automation for data discovery, data life cycle management, data quality and productivity control while accelerating analytics agility.

What you’ll be doing

The role’s direct responsibilities include:

• Implement data ingestion and data integration solutions to support Stream’s simulation platform on AWS as part of a data mesh strategy.
• Manage and build ETL processes, extraction & integration of data from various data sources enabling a continuous data value stream.
• Oversee data quality monitoring, including processing, management and cleaning of data flows leveraged for analytics purposes while ensuring federated governance.
• Create custom data models for automated analysis, training machine learning, and enabling embedded BI reporting with row level security.
• Leverage best practices and best-in-class products and services to build cost-effective, scalable, and reliable data pipelines, data domains and data consumer API’s.
• Define and manage relational and distributed semi-structured NoSQL data warehouses and schemas that form the core of data mesh or fabric architectures.
• Translate data requirements into detailed designs and pipeline architecture, ensuring proper documentation of methodologies including deployable data services with a data mesh.
• Define best practices around data lineage, versioning, services deployment, extraction, modeling, consumption & governance at Stream.

Requirements

To ensure success in the role, you will need the following:

• Experience and formal qualifications in a STEM related discipline, such as Engineering or Computer Science, or certification and extensive work experience in a data centric development role with an understanding of how to implement data system architectures (distributed data mesh preferred)
• Experience designing and implementing data pipeline solutions accessing data from big data frameworks (AWS, GCP BigQuery) as well as hybrid architectures with on-premise data connectors (SQL Server, PostgreSQL, Historian, Timeseries) 
• Data stack working proficiency in SQL, Python, Java, and API development utilizing GraphQL. Experience with messaging platforms like Kafka or RabbitMQ, data ingestion tools such as Airbyte, DBT, Databricks, MLflow an asset.
• Experience in data analytics engineering, orchestrating data pipelines, performance tuning, and script and test automation development.
• Experience architecting ETL solutions for a wide variety of sources, data abstractions, and data pipelines for real-time, streaming, batch, and on-demand workloads.
• Experience with data security strategies, configuring security for reporting, and certifications such as SOC 2.

Work environment/Hours/Location

This is a full-time position based on a 40-hour work week. We are driven by objectives, however, not timesheets. Stream’s head office is location in Calgary, Alberta with remote workers located across the country in British Columbia, Ontario, and Quebec. We are happy to provide a hybrid work environment, enabling you to work remotely as needed. We encourage a strong collaborative culture and provide workspaces in office for team collaboration, design, planning sessions and social activities.

Benefits

Company benefits are available to our full-time, permanent employees and include extended health care, dental, long-term disability, AD&D, and life insurance for you and your dependents. The real benefits, in our opinion exist in the ability to become part of a flexible, passionate, and dedicated team where we continue to learn from one another every day. If you love that small-company feel with big-company goals, Stream just might be a fit for you.

Come do great things with us!