Principal Datadog Engineer
Gdh Consulting, Inc.

We are supporting a high-visibility federal program dedicated to modernizing software infrastructure for a key governmental department. This role focuses on establishing and scaling enterprise-wide observability using Datadog. The Principal Datadog Engineer will serve as the technical authority in designing, implementing, and governing complex monitoring solutions, influencing platform architecture, and guiding organizations toward best practices in observability. This position offers an opportunity to shape enterprise monitoring strategies, contribute to a cutting-edge tech environment, and lead critical technical initiatives.

Key Responsibilities:

• Define and drive high-level observability strategies, addressing unique and complex challenges that impact system design and future innovations.
• Architect and implement scalable Datadog solutions-dashboards, monitors, alerts, APM, synthetic testing, and log analysis-to enhance visibility across applications, infrastructure, and security layers.
• Lead large-scale enterprise integrations and oversee the deployment of Datadog across multiple systems, ensuring consistency, reliability, and security.
• Identify workflow inefficiencies and develop tools or methodologies to improve operational productivity, system reliability, and cost efficiency.
• Serve as the escalation point for advanced Datadog performance and monitoring issues, performing root cause analysis with expertise in tracing, logs, and dashboards.
• Mentor and develop engineering teams, promoting best practices and fostering continuous improvement through training and documentation.
• Collaborate with cross-functional teams-including DevOps, SRE, Cloud, Security, and Application Development-to embed observability into all facets of operations.
• Influence architecture decisions and lead technical governance efforts, ensuring alignment with program goals and industry standards.
• Communicate complex technical concepts effectively to non-technical stakeholders, representing the organization's technical capabilities externally when needed.

Qualifications:

• Bachelor's Degree and a minimum of 10 years of relevant experience, or 14+ years of relevant experience in lieu of a degree.
• Proven expertise in large-scale, enterprise-level software environments with deep, recent experience in Datadog architecture, configuration, and optimization.
• Datadog Fundamentals Certification (or willing to obtain) and hands-on proficiency with dashboards, APM, synthetic tests, and integrations.
• Strong engineering background with proficiency in Python, Bash, PowerShell, YAML; experience building complex automation scripts and Lambda functions.
• Demonstrated success in implementing infrastructure and application monitoring solutions using Datadog under the hood.
• Cloud experience, preferably AWS, with Infrastructure-as-Code tools such as Terraform.
• Prior experience as a Cloud DevOps or Automation Engineer with a track record of delivering scalable observability solutions.
• Excellent technical communication skills, capable of explaining complex concepts to diverse audiences.
• Analytical mindset with strong problem-solving skills and a strategic approach to future-proofing observability capabilities.
• Ability to lead initiatives independently, influencing architecture and operational practices at an enterprise level.

Nice to Have:

• Additional Datadog certifications or willingness to obtain certifications.
• Experience with DevSecOps practices and integrating security monitoring into observability platforms.
• Background in modern, innovative engineering environments (e.g., high-tech companies known for rapid development cycles).

Preferred Skills (Not required):

• Experience scripting in Python for automation, monitoring, and alerting solutions.
• Exposure to advanced security automation and monitoring practices.
• Previous work in highly modernized, cutting-edge technology settings.

Ability to obtain public trust clearance (Higher clearances are acceptable but not required).
Must be legally authorized to work in the United States without employer sponsorship, now or in the future.

Pay Rate: $100.00 - $105.00 Hourly

About GDH:
At GDH, we believe in the power of people and the importance of caring. Our culture statement, "We care about people," isn't just a tagline - it's the core of everything we do. GDH is a premier staffing and talent solutions company dedicated to helping businesses find the best talent and assisting job seekers in finding their dream jobs.

Who We Are:
GDH, founded in 2001, has grown into a leader in providing staffing solutions across various industries. We specialize in IT across several sectors, connecting top talent with leading enterprises. As a Best of Staffing firm recognized for excellence in client, employee, talent, and women's services, we pride ourselves on our commitment to quality and service.

GDH Benefits

GDH offers a range of employee benefits that are designed to promote well-being and help maintain a healthy work-life balance. These comprehensive benefits cover various aspects of an employee's life and aim to enhance their overall experience with the company. Our health benefits include three medical insurance options with access to KISx Card, Zero Card, and HealthJoy concierge services. Other plan offerings include dental, vision, life, disability, supplemental insurance, and pet insurance plans. Enjoy additional perks like holiday pay, 401(k) plan, direct deposit, an employee referral program, work-life balance benefits, a Wellbeats membership, a discounted gym membership program, and more! For more detailed information on benefits, please go to GDH's website under the tab for candidates.

GDH provides equal employment opportunities (EEO) to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, gender, sex (including pregnancy), sexual orientation, gender identity, national origin, age, disability, genetic information, marital status, ancestry, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable federal, state, and local laws. Applicants with disabilities who require an accommodation or assistance in applying and/or for interviewing, please contact our HR Department.

Please visit our notice of collection for California applicants.

---