Most WordPress sites that care about speed sit behind a full-page cache: a caching plugin, the host's edge cache or a CDN. That cache is a problem for popup A/B tests. The HTML a visitor receives was rendered for someone else, possibly hours ago, so it can't carry that visitor's variant, a fresh nonce or any decision that depends on who they are. Popup plugins often break here quietly: variants stick to the cached page instead of the person, views are counted on page load, and a "winner" is announced after 40 visits. When we built DEV3 AI Popup & Lead Capture Builder, we wanted A/B results a marketer could act on, on exactly these cached sites. This post explains how the work is split between the server, the browser and the database to get there.
Illustration: popups, slide-ins, bars and phone-friendly sheets turning visitors into leads.
What it does and who it's for
DEV3 AI Popup & Lead Capture Builder makes lead capture campaigns in five formats: centered popups, full-screen welcome mats, corner slide-ins, floating bars and inline forms. It comes with 12 templates, six triggers, page and visitor targeting, frequency caps, A/B tests with up to four variants, a leads table with CSV export, and delivery to Mailchimp, Brevo or a signed webhook. DEV3 AI, using your own Google Gemini API key, writes copy, proposes offer variants for a test, suggests targeting, explains results and drafts whole campaigns. It is built for:
- Bloggers and publishers growing a newsletter with content upgrades and inline forms after posts, without slowing pages down.
- WooCommerce stores that want an exit-intent offer, a free-shipping bar or a first-order coupon, targeted by cart value and delivered as a unique one-time code.
- Marketers who test. They need to know whether "free guide" beats "10% off" and can't wait for a developer to set up a split test.
- Sites that take mobile seriously. Mobile-safe mode turns popups into non-blocking bottom sheets on phones by default.
How to set it up
- Unzip the bundle, then upload
dev3-ai-popup-builder.zipunder Plugins → Add New → Upload Plugin and activate it. The DEV3 AI Popups menu appears. - Optional: under API Config, paste a Google Gemini API key and click Validate & save. Everything except the DEV3 AI drafts works without a key.
- Optional: under Integrations, add a Mailchimp or Brevo key and click Test connection, or set a webhook signing secret.
- In Settings, check the visitor experience defaults (mobile-safe mode, one overlay per page view, roles to hide from) and write your brand voice.
- Click Campaigns → New campaign and start from a template or a one-sentence description.
New campaign: 12 templates by format, or "Describe with DEV3 AI" for a drafted campaign.
How to use it
Launch a campaign. Pick a template and work through the editor tabs: Content, Form & offer, Design, Triggers and Targeting. The preview uses the real front-end renderer. Preview on site shows the draft on a real page to you only, and Publish puts it live.
Test two offers. On the A/B test tab, add a copy of the control, or click Generate variants with DEV3 AI for three ideas with different angles and Add to test the ones you like. Set the weights and publish. Each visitor keeps the same variant on every page.
Decide. The Results tab shows views, leads, conversion rate, the change against the control and a confidence figure for each variant. Explain the results gives a plain-language summary. Declare winner stops the test and shows that variant to everyone.
The Results tab with demo data: variant B leads with 99.5% confidence after about 1,900 views per variant.
The rest of this post is about where those numbers come from, and why they still mean something on a cached site.
The problem: four questions a cache gets in the way of
An A/B test needs four answers for every visitor: should this campaign run here, which variant does this person see, did they actually see it, and did they convert. A typical plugin answers all four in PHP while rendering the page. On a cached site:
- The page rules are fine, because they depend only on the URL, and the cached HTML is for that URL.
- The variant is wrong, because it was chosen for whoever warmed the cache.
- "Seen" is wrong, because the page was rendered rather than the popup shown, and many popups never open.
- "Converted" is open to abuse, because a public counter endpoint with a stale nonce either rejects everyone or has to accept anything.
So we split the four questions between three places. The server answers what only depends on the URL. The browser answers what depends on the visitor. The database counts only events the server has checked.
What the server decides, and what it hands over
Page rules (front page, post types, specific posts, taxonomy terms, URL wildcards, 404 and search results) are evaluated in PHP by Page_Rules::matches(). Only campaigns that pass are printed into a small JSON boot object. Pages with no matching campaign load no script and no stylesheet at all. A FrontendTest case checks this.
The payload holds exactly what the browser needs to decide the rest, and nothing more. Frontend::public_campaign() copies the variants, form fields, triggers and the visitor-side targeting rules (devices, referrer, UTM, login, new or returning, pages viewed, cart), and leaves out the integration settings and any coupon or reveal code. A PHPUnit test encodes the payload and asserts that neither a reveal code nor a webhook URL appears in it. public_campaign() also hands over only the variants that are serving. During a test, that means every variant with a weight above 0. After a winner is declared, it is that variant alone, and otherwise the control.
So a variant set to weight 0 drops out of the test without being deleted. When a page cache still holds an older payload, the browser logic below copes with either version.
Sticky variants without cookies
The variant is chosen in the browser, once per visitor and campaign, and stored in localStorage under dev3pu_ab_<id>:
export function pickVariant( variants, stored, rand ) {
const list = variants || [];
if ( ! list.length ) {
return 'a';
}
if ( stored && list.some( ( v ) => v.id === stored ) ) {
return stored;
}
const total = list.reduce( ( s, v ) => s + Math.max( 0, v.weight || 0 ), 0 );
if ( total <= 0 ) {
return list[ 0 ].id;
}
let x = rand * total;
for ( const v of list ) {
x -= Math.max( 0, v.weight || 0 );
if ( x < 0 ) {
return v.id;
}
}
return list[ list.length - 1 ].id;
}
(from src/frontend/engine.js)
It is a pure function with the random number passed in, so Jest can test it exactly: with 50/50 weights 0.1 gives A, 0.9 gives B, and a stored B stays B. A stored id that no longer exists is picked again rather than trusted. That matters after Declare winner: the page serves one variant, the old assignment no longer matches, and the visitor sees the winner.
Why not a cookie? A cookie set by PHP lives in the cached response. One set by JavaScript would be sent with every request, and some caches bypass pages that carry unknown cookies. localStorage stays in the browser. Every read and write is wrapped in try/catch, so a private window that throws just gets a fresh random pick.
Inline forms: choose before the first paint
Overlays are easy: nothing exists until a trigger fires, so the variant is chosen then. Inline forms are part of the cached HTML, and swapping them after load would cause layout shift and a flicker of the wrong copy.
The renderer prints every serving variant. All but the first get the hidden attribute, and a tiny inline script runs straight after the markup:
(function(){try{
var id=42,w={"a":50,"b":50},k="dev3pu_ab_"+id,v=localStorage.getItem(k);
if(!v||!(v in w)){ /* weighted pick */ localStorage.setItem(k,v); }
var n=document.querySelectorAll('[data-dev3pu-id="'+id+'"][data-dev3pu-inline]');
for(var i=0;i<n.length;i++){ n[i].hidden=n[i].getAttribute("data-dev3pu-variant")!==v; }
}catch(e){}})();
(condensed from Renderer::inline; the campaign id and weights are filled in by PHP)
Because it runs synchronously before the browser paints that part of the page, the visitor only ever sees their own variant. If storage throws, the catch leaves variant A visible, which is the control, so the worst case is a fair default. A FrontendTest case checks that exactly one variant is printed without hidden and that the picker script is present.
What counts as a view
A view is counted when the visitor could actually see the campaign, not when the page loads:
- Overlays send
viewfromshow(), after the trigger fired, the frequency caps and the one-overlay-per-page-view guard allowed it, and the DOM was built. - Inline forms send
viewonce at least half the form has scrolled into the viewport, using anIntersectionObserverthat disconnects after the first hit.
Both go through one track() helper, which uses fetch with keepalive: true so the request survives the visitor clicking away. Failures are ignored, because analytics must never break the page.
Counting on page load would mix "saw the offer and said no" with "left before the exit trigger fired", which dilutes every variant's rate and hides real differences.
What counts as a conversion
The public endpoint POST /dev3pu/v1/public/track has no nonce, because a nonce in cached HTML goes stale and then rejects everyone. Instead its permission callback rejects requests whose Origin header names another host, and applies a per-IP rate limit of 120 events a minute. The handler then trusts very little:
$campaign = Campaigns::get( (int) $request['campaign_id'] );
$variant = substr( preg_replace( '/[^a-d]/', '', strtolower( (string) $request['variant'] ) ), 0, 1 );
if ( ! $campaign || 'active' !== $campaign['status'] || ! in_array( $variant, wp_list_pluck( $campaign['config']['variants'], 'id' ), true ) ) {
return $this->respond( new \WP_Error( 'dev3pu_not_found', __( 'Unknown campaign.', 'dev3-ai-popup-builder' ), array( 'status' => 404 ) ) );
}
$event = (string) $request['event'];
if ( 'click' === $event ) {
// A click on a CTA button is the conversion of a campaign without a form.
if ( 'cta' !== $campaign['config']['form']['mode'] ) {
return $this->respond( array( 'ok' => true ) );
}
$event = 'convert';
}
Stats::bump( $campaign['id'], $variant, $event );
(condensed from Public_Controller::track)
The browser can't send convert at all. The route's schema accepts only view, close and click. A click becomes a conversion only for button-only campaigns, where the click is the goal. For form campaigns, conversions are counted somewhere else: in Capture::submit(), after validation, the honeypot, the minimum fill time and the per-IP submit limit of 10 a minute, and only when Leads::upsert() reports a new lead. The same email submitted twice to the same campaign updates the lead without counting a second conversion, and submissions from Preview on site are never counted. RestTest checks each branch: an unknown variant returns 404, an unknown event 400, a draft campaign 404, and a click on a form campaign adds no conversion.
Counting without a read-modify-write
Each event becomes one statement:
"INSERT INTO {$table} (campaign_id, variant, day, {$col}) VALUES (%d, %s, %s, 1)
ON DUPLICATE KEY UPDATE {$col} = {$col} + 1"
(from Stats::bump; $col comes from a fixed whitelist of views, conversions and closes)
The primary key is (campaign_id, variant, day), so the table holds at most four rows per campaign per day, however busy the site is, and per-variant totals are a SUM … GROUP BY variant over a few hundred rows. We didn't store raw events. That would grow without limit and add little to an A/B decision, because daily granularity is all the "Leads per day" chart needs. Unknown event names return false before any SQL runs.
Illustration: visitors pass the page cache, split into sticky variants, and are counted and weighed fairly.
From counters to a verdict
Stats::results() compares each variant with the control (the first variant). It reports the conversion rate, the relative uplift and a two-sided confidence from a pooled two-proportion z-test:
public static function confidence( $n1, $c1, $n2, $c2 ) {
if ( $n1 <= 0 || $n2 <= 0 ) {
return 0.0;
}
$p1 = $c1 / $n1;
$p2 = $c2 / $n2;
$p = ( $c1 + $c2 ) / ( $n1 + $n2 );
$se = sqrt( $p * ( 1 - $p ) * ( 1 / $n1 + 1 / $n2 ) );
if ( $se <= 0 ) {
return 0.0;
}
$z = abs( $p2 - $p1 ) / $se;
return round( 100 * self::erf( $z / M_SQRT2 ), 1 );
}
(from Stats::confidence)
PHP has no built-in error function, so Stats::erf() uses the Abramowitz–Stegun 7.1.26 approximation, which is accurate to about 1.5 × 10⁻⁷. A test checks erf(1) against 0.8427.
Take the demo data in the screenshot above. A had 96 leads from 1,861 views (5.16%) and B had 144 from 1,953 (7.37%). The pooled rate is 240 / 3,814 = 6.29%, the standard error 0.79 percentage points, and z = 2.82. That gives 99.5% confidence, and uplift is (7.37 − 5.16) / 5.16 = +42.8%. Two of the test fixtures show how much sample size matters: 50 versus 75 conversions from 1,000 views each comes out near 98%, while 5 versus 6 from 100 views each stays below 50%.
Two rules stop the table from crying wolf:
- A variant is only marked significant (its confidence turns green) at 95% confidence and with at least 100 views on both it and the control. The "Leading" badge only marks the highest rate so far.
enough_datais false until every variant has 100 views. Explain the results passes the numbers to DEV3 AI for a plain-language summary, but the server blanks any winner the model names whileenough_datais false. An AiTest case with a mocked response proves it: the model says "b", and the plugin returns no winner.
Declare winner writes ab = { enabled: false, winner: 'b' } and leaves the stats rows untouched, so the history stays on the Results tab while everyone sees B.
Trade-offs we accepted
- Per browser, not per person. A visitor on a phone and a laptop can land in both variants. Avoiding that needs an identity most visitors to a lead form don't have yet.
- Views are displays, not unique visitors. With the frequency set to "every page view", one person can add several views. The default cap is once every 7 days, and never again after a sign-up. Every variant runs under the same cap, so the comparison is fair, but the numbers are not unique-visitor counts.
- A fixed-horizon test, read continuously. A z-test assumes you look once. The Results tab can be opened every day, and repeated looks raise the false-positive rate. The 100-view floor on every variant and the 95% bar reduce the risk but don't remove it, so we say "leading", not "winner", until a person decides.
- Tracking can be switched off. With Count views, closes and conversions off, campaigns still run but A/B results stay empty. That setting is for sites that want no analytics at all.
How it was tested
The plugin has 69 PHPUnit tests, run in wp-env, and 59 Jest tests. For this feature they cover weighted and sticky variant choice, the pre-paint inline picker, the public payload leaving out codes and integrations, every track branch, conversions counted only on new leads, cross-origin rejection and rate limits, the z-test and erf, and the AI explanation being unable to name a winner early. The DEV3 AI paths were tested with mocked API responses, and Mailchimp and Brevo against mocks of their documented APIs. A run with a real key and live accounts is still to come.
Mobile-safe mode on a phone: the popup becomes a bottom sheet, and its views are counted only when it actually opens.
DEV3 AI Popup & Lead Capture Builder ships in the DEV3 AI Forms & Lead Generation Plugins Bundle on the DEV3 marketplace, alongside our Form Builder, Entry Insights and Form Autofill & Spam Shield plugins, and is being prepared for CodeCanyon. The architecture, the AI drafting features and what we verified are in the case study.
