The Role Context:
This is an exciting opportunity to join a dynamic and growing organization,
working at the forefront of technology trends and developments in social impact
sector. Wadhwani Center for Government Digital Transformation (WGDT) works with
the government ministries and state departments in India with a mission of “Enabling
digital transformation to enhance the impact of government policy, initiatives
and programs”.
We are seeking a highly motivated and
detail-oriented individual to join our team as a Data Engineer with
experience in the designing, constructing, and maintaining the architecture and
infrastructure necessary for data generation, storage and processing and
contribute to the successful implementation of digital government policies and
programs. You will play a key role in developing, robust, scalable, and
efficient systems to manage large volumes of data, make it accessible for
analysis and decision-making and driving innovation & optimizing operations
across various government ministries and state departments in India.
Key Responsibilities:
a. Data Architecture Design: Design, develop, and maintain scalable data pipelines and
infrastructure for ingesting, processing, storing, and analyzing large volumes
of data efficiently. This involves understanding business requirements and
translating them into technical solutions.
b. Data Integration: Integrate data from various sources such as databases, APIs, streaming
platforms, and third-party systems. Should ensure the data is collected
reliably and efficiently, maintaining data quality and integrity throughout the
process as per the Ministries/government data standards.
c. Data Modeling: Design and implement data models
to organize and structure data for efficient storage and retrieval. They use
techniques such as dimensional modeling, normalization, and denormalization
depending on the specific requirements of the project.
d. Data Pipeline Development/ ETL (Extract,
Transform, Load): Develop data pipeline/ETL processes to
extract data from source systems, transform it into the desired format, and
load it into the target data systems. This involves writing scripts or using
ETL tools or building data pipelines to automate the process and ensure data
accuracy and consistency.
e. Data Quality and Governance: Implement data quality checks and data governance policies to ensure
data accuracy, consistency, and compliance with regulations. Should be able to design
and track data lineage, data stewardship, metadata management, building business
glossary etc.
f. Data lakes or Warehousing: Design and maintain data lakes and data warehouse to store and manage
structured data from relational databases, semi-structured data like JSON or
XML, and unstructured data such as text documents, images, and videos at any
scale. Should be able to integrate with big data processing frameworks such as
Apache Hadoop, Apache Spark, and Apache Flink, as well as with machine learning
and data visualization tools.
g. Data Security: Implement security practices, technologies, and policies designed to
protect data from unauthorized access, alteration, or destruction throughout
its lifecycle. It should include data access, encryption, data masking and
anonymization, data loss prevention, compliance, and regulatory requirements
such as DPDP, GDPR, etc.
h. Database Management: Administer and optimize databases, both relational and NoSQL, to manage
large volumes of data effectively.
i. Data Migration: Plan and execute data migration projects to transfer data between
systems while ensuring data consistency and minimal downtime.
a. Performance Optimization: Optimize data pipelines and queries for performance and scalability. Identify
and resolve bottlenecks, tune database configurations, and implement caching
and indexing strategies to improve data processing speed and efficiency.
b. Collaboration: Collaborate with data scientists, analysts, and other stakeholders to
understand their data requirements and provide them with access to the
necessary data resources. They also work closely with IT operations teams to
deploy and maintain data infrastructure in production environments.
c. Documentation and Reporting: Document their work including data models, data pipelines/ETL
processes, and system configurations. Create documentation and provide training
to other team members to ensure the sustainability and maintainability of data
systems.
d. Continuous Learning: Stay updated with the latest technologies and trends in data engineering
and related fields. Should participate in training programs, attend
conferences, and engage with the data engineering community to enhance their
skills and knowledge.
Desired Skills/ Competencies
KPIs:
Years of experience of the current role
holder
New Position
Ideal years of experience
3 – 5 years
Career progression for this role
CTO WGDT (Head of Incubation Centre)
*******************************************************************************
Wadhwani
Corporate Profile: (Click on this link)
Our Culture:
WF is a global
not-for-profit, and works like a start-up, in a fast-moving,
dynamic pace where change is the only constant and flexibility is the key to
success.
Three mantras that we
practice across job roles, levels, functions, programs and initiatives, are
Quality, Speed, Scale, in that order.
We are an ambitious and
inclusive organization, where everyone is encouraged to contribute and
ideate. We are intensely and insanely focused on driving excellence in
everything we do.
We want individuals with
the drive for excellence, and passion to do whatever it takes to deliver
world class outcomes to our beneficiaries. We set our own standards often more
rigorous than what our beneficiaries demand, and we want individuals who love
it this way.
We
have a creative and highly energetic environment – one in which we look to each
other to innovate new solutions not only for our beneficiaries but for
ourselves too. Open to collaborate with a borderless mentality, often going
beyond the hierarchy and siloed definitions of functional KRAs, are the
individuals who will thrive in our environment.
This
is a workplace where expertise is shared with colleagues around the
globe. Individuals uncomfortable with change, constant innovation, and
short learning cycles and those looking for stability and orderly working days
may not find WF to be the right place for them.
Finally,
we want individuals who want to do greater good for the society leveraging
their area of expertise, skills and experience.
The foundation is an equal
opportunity firm with no bias towards gender, race, colour, ethnicity, country,
language, age and any other dimension that comes in the way of progress.
Join
us and be a part of us!
Want to receive frequent updates by email? Subscribe to our automatic job service!
Company:
Wadhwani FoundationEmployee Type:
Full timeLocation:
IndiaSalary:
$ 49218 - $ 91405