Senior Manager Experimentation, Testing, & Learning

Hiring Locations:

Addison – Atlanta – Autin – Boston – Chicago – Columbus – Dallas – Denver – Houston – Huntsville – Jersey City – Kirkland – Los Angeles – Minneapolis – Mountain View – Nasville – New York – Orlando – Philadelphia – Phoenix – Pleasnton – Portland – Releigh – San Diego – San Francisco – Santa Clara – Seattle – Vienna – Waltham – Washington, D.C. – West Palm Beach

The Senior Manager, Experimentation, Testing, & Learning owns the quarterly experimentation roadmap and learning agenda that drives measurable improvements in buying group progression. This role runs end-to-end test governance across buying group experience, ensuring tests are designed well, implemented correctly, measured cleanly, and translated into clear scale or stop decisions.

What you will own

Quarterly experimentation roadmap, learning agenda, and test portfolio aligned to brand, events, and integrated program goals with emphasis on buying group progression.
Test intake, prioritization, design standards, and readout templates used across teams.
Scaling criteria and playbooks for rolling out tests that show measurable lift.
Governance for experimentation processes, including decision rules and documentation standards.
Responsibilities

Own and evolve the quarterly experimentation roadmap and learning agenda, aligned to brand, events, and integrated program goals with emphasis on buying group progression.
Manage test intake and prioritization using clear criteria, expected impact, feasibility, time-to-learn, and cross-channel applicability, maintaining balanced coverage across paid, owned, site, and creative.
Define and maintain test design standards: hypotheses, primary and secondary KPIs, success thresholds, guardrails, and decision rules.
Partner with Measurement to define incrementality approaches, holdouts, lift studies, geo tests, platform experiments and standardize efficiency metrics.
Partner with agency and platform teams to ensure clean implementation, consistent exposure rules, and strong QA across channels.
Coordinate with DX on site and landing page tests, including experimentation tooling, tagging, data layer dependencies, and conversion-path measurement.
Partner with creative and agency teams to structure creative tests (message, offer, asset variants) with clear learning objectives and measurement plans.
Standardize how results are reported — readout templates that clearly summarize lift, confidence, segment impacts, tradeoffs, and recommended decisions.
Lead quarterly readouts that drive decisions to scale, stop, iterate, or retest, with documented rationale.
Maintain scaling criteria and update global playbooks when tests show measurable lift and repeatability.

Qualifications
7+ years in growth marketing, performance marketing, experimentation, analytics, or marketing operations with ownership of cross-channel testing programs and a track record of turning test results into strategic decisions.
Hard Skills

Experimentation Design: A/B, multivariate, holdouts, lift studies, geo experiments and common design pitfalls to avoid
Cross-Channel Testing: paid media (programmatic, search, social), owned channels, site experience, and creative
Measurement Partnership: incrementality design, impact evaluation, and efficiency metric standardization
Agency & Platform Execution: clean test setup, QA, trafficking, and consistent measurement across partners
Reporting & Decision Frameworks: standardized readouts that drive repeatable scaling decisions
Soft Skills (Leadership Competencies)

Strategic Vision: able to define a learning agenda that connects test-and-learn activity to business outcomes, not just optimization metrics
Cross-Functional Influence: drives alignment and adoption of experimentation standards across Media, Measurement, DX, Integrated Marketing, and agency partners
Decisiveness & Clarity translates ambiguous test results into clear, actionable recommendations for senior stakeholders
Accountability & Rigor holds the bar on test design quality and documentation standards across all partners
Structured Communication: turns complex measurement concepts into digestible readouts that inform strategic decisions
Preferred

Experience tying experimentation to buying group progression or multi-stakeholder B2B journeys
Experience with onsite experimentation tools and landing page optimization programs
Familiarity with media platform experimentation products — conversion lift, brand lift, incrementality testing

For positions in this location, we offer a base pay of $155,400 – $272,000, plus equity (when applicable), variable/incentive compensation and benefits. Sales positions generally offer a competitive On Target Earnings (OTE) incentive compensation structure. Please note that the base pay shown is a guideline, and individual total compensation will vary based on factors such as qualifications, skill level, competencies, and work location. We also offer health plans, including flexible spending accounts, a 401(k) Plan with company match, ESPP, matching donations, a flexible time away plan and family leave programs. Compensation is based on the geographic location in which the role is located and is subject to change based on work location.