When a Demo Fails (And Why That’s Valuable)
When a Demo Fails (And Why That’s Valuable)
We tested an interactive Jira Software demo to improve signups. While it didn’t increase conversions, it revealed valuable insights that shaped future onboarding experiments.
Product
Jira Software
What I did
Product Design
my role
design lead
Timeline
6 Weeks
The Why
The Why
Jira Software struggled to communicate its value to new users before signup. Existing tours were static or buried, leading to low engagement and high drop-offs. A previous experiment adding interactivity to feature pages had shown a ~10% uplift in signups and stronger first-week engagement. Building on that success, the team asked: could we design an interactive, hands-on demo that let users try Jira before creating an account?
Jira Software struggled to communicate its value to new users before signup. Existing tours were static or buried, leading to low engagement and high drop-offs. A previous experiment adding interactivity to feature pages had shown a ~10% uplift in signups and stronger first-week engagement. Building on that success, the team asked: could we design an interactive, hands-on demo that let users try Jira before creating an account?
The Why
Jira Software struggled to communicate its value to new users before signup. Existing tours were static or buried, leading to low engagement and high drop-offs. A previous experiment adding interactivity to feature pages had shown a ~10% uplift in signups and stronger first-week engagement. Building on that success, the team asked: could we design an interactive, hands-on demo that let users try Jira before creating an account?
Context & Constraints
Context & Constraints
Audience: Mix of existing and new users (confounded results).
KPIs: Demo starts/completions; downstream: signup/activation.
Constraints: Placement limits; lightweight build for quick test.
Audience: Mix of existing and new users (confounded results).
KPIs: Demo starts/completions; downstream: signup/activation.
Constraints: Placement limits; lightweight build for quick test.
Context & Constraints
Audience: Mix of existing and new users (confounded results).
KPIs: Demo starts/completions; downstream: signup/activation.
Constraints: Placement limits; lightweight build for quick test.
Impact
Impact
Outcome: After 14 days, rates remained flat—but the experiment surfaced critical insights to guide our next optimizations.
Outcome: After 14 days, rates remained flat—but the experiment surfaced critical insights to guide our next optimizations.
Impact
Outcome: After 14 days, rates remained flat—but the experiment surfaced critical insights to guide our next optimizations.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
0%Account
Creation0%Viewed Email
OTP Screen-0%Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
Treatment A
Account
Creation
Viewed Email
OTP Screen
Signups
Changes:
Moved the product shot to the right and the redesigned signup form to the left.
Increased the size and prominence of the product shot.
Outcome: Despite positive interaction metrics, the overall signup rate decreased significantly compared to the control.
Treatment B
Account
Creation
Viewed Email
OTP Screen
Signups
Changes:
Same setup as Treatment A with two additional elements:
An animated product shot to enhance engagement.
Social proof below the signup form.
Outcome: Smaller signup rate decrease (not statistically significant) compared to the control.
What we learned
What we learned
Analysis revealed our demo underperformed due to audience misalignment and low visibility—next, we’ll refine our targeting and boost demo placement to maximize reach and impact.
Analysis revealed our demo underperformed due to audience misalignment and low visibility—next, we’ll refine our targeting and boost demo placement to maximize reach and impact.
What we learned
Analysis revealed our demo underperformed due to audience misalignment and low visibility—next, we’ll refine our targeting and boost demo placement to maximize reach and impact.
Process
Process
This overview spans initial research through final execution and illustrates how I crafted an engaging interactive demo. The process prioritized user needs and delivered tangible results by leveraging data-driven insights and collaborative workshops.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.
Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
This overview spans initial research through final execution and illustrates how I crafted an engaging interactive demo. The process prioritized user needs and delivered tangible results by leveraging data-driven insights and collaborative workshops.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Process
This overview spans initial research through final execution and illustrates how I crafted an engaging interactive demo. The process prioritized user needs and delivered tangible results by leveraging data-driven insights and collaborative workshops.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Attractiveness Analysis
To measure the success of the redesign, I used attractiveness analysis—assessing how visually appealing and engaging each element was for users. This helped determine whether the updated layout effectively captured and retained their interest.
Before the Redesign
The original hero section prioritized an embedded signup form, overshadowing the product shot. While functional, it didn’t draw users into the product visuals, reducing the overall appeal of the hero section.Engagement Rate Analysis
To gauge the redesign’s effectiveness, I analyzed engagement rates—tracking how users interacted with core elements on the page before and after the update. This metric revealed whether the new layout successfully captured user interest and encouraged deeper interaction.
Before the Redesign: While the original hero section’s embedded signup form had a high click rate, the product shot and other key information saw minimal engagement. This indicated that users were focused on signing up but missing valuable details about the product.
Final Design
The final interactive demo allowed users to explore Jira Software’s board and issue features in a hands-on, guided format, seamlessly merging product functionality with marketing for an authentic experience.
Challenges:
Despite strong engagement, the demo did not reach the 5% signup goal.
Key obstacles included targeting a broad audience and the demo’s low-visibility placement.
Final Design
The final interactive demo allowed users to explore Jira Software’s board and issue features in a hands-on, guided format, seamlessly merging product functionality with marketing for an authentic experience.
Challenges:
Despite strong engagement, the demo did not reach the 5% signup goal.
Key obstacles included targeting a broad audience and the demo’s low-visibility placement.
Final Design
The final interactive demo allowed users to explore Jira Software’s board and issue features in a hands-on, guided format, seamlessly merging product functionality with marketing for an authentic experience.
Challenges:
Despite strong engagement, the demo did not reach the 5% signup goal.
Key obstacles included targeting a broad audience and the demo’s low-visibility placement.
Final Design
The final interactive demo allowed users to explore Jira Software’s board and issue features in a hands-on, guided format, seamlessly merging product functionality with marketing for an authentic experience.
Challenges:
Despite strong engagement, the demo did not reach the 5% signup goal.
Key obstacles included targeting a broad audience and the demo’s low-visibility placement.
Reflection
Lessons Learned
Target Audience:
Challenge: The experiment targeted all users visiting the product tour, but 30% were already logged in and not in the target demographic.
Lesson: Narrow the target audience to new users (N2N) for future experiments.
Visibility:
Challenge: The demo was placed below the fold, with only 37% of users scrolling to view it.
Lesson: Ensure interactive elements are visible above the fold to maximize exposure.
Data Dilution:
Challenge: The exposure event fired at page load, including users who never interacted with the demo.
Lesson: Redefine exposure criteria to reflect actual interaction.
Reflection
Lessons Learned
Target Audience:
Challenge: The experiment targeted all users visiting the product tour, but 30% were already logged in and not in the target demographic.
Lesson: Narrow the target audience to new users (N2N) for future experiments.
Visibility:
Challenge: The demo was placed below the fold, with only 37% of users scrolling to view it.
Lesson: Ensure interactive elements are visible above the fold to maximize exposure.
Data Dilution:
Challenge: The exposure event fired at page load, including users who never interacted with the demo.
Lesson: Redefine exposure criteria to reflect actual interaction.
Reflection
Lessons Learned
Target Audience:
Challenge: The experiment targeted all users visiting the product tour, but 30% were already logged in and not in the target demographic.
Lesson: Narrow the target audience to new users (N2N) for future experiments.
Visibility:
Challenge: The demo was placed below the fold, with only 37% of users scrolling to view it.
Lesson: Ensure interactive elements are visible above the fold to maximize exposure.
Data Dilution:
Challenge: The exposure event fired at page load, including users who never interacted with the demo.
Lesson: Redefine exposure criteria to reflect actual interaction.
Reflection
Lessons Learned
Target Audience:
Challenge: The experiment targeted all users visiting the product tour, but 30% were already logged in and not in the target demographic.
Lesson: Narrow the target audience to new users (N2N) for future experiments.
Visibility:
Challenge: The demo was placed below the fold, with only 37% of users scrolling to view it.
Lesson: Ensure interactive elements are visible above the fold to maximize exposure.
Data Dilution:
Challenge: The exposure event fired at page load, including users who never interacted with the demo.
Lesson: Redefine exposure criteria to reflect actual interaction.
More Works More Works
More Works More Works
bildkritik
Go Back To Top
bildkritik
Go Back To Top