Pivotal AppsManager Integrations

After my 2nd year of undergrad at Berkeley in the summer of 2019, I was a product manager intern at Pivotal Software. I worked on the AppsManager team within Cloud R&D. AppsManager is the developer facing UI for Pivotal Application Service (PAS). It enables enterprise app teams to discover, connect, and apply the full functionality of PAS.

I focused on identifying and tracking developer facing features of PAS and PAS adjacent teams and enabling their integration into AppsManager within 1 release.

Challenges

Understanding

Not only did I need to understand my product (appsmanager) fully, but I also needed to gain a high level understanding of what other teams do - the scope of their product and how it interacts with apps manager. That way, I could discover integration points and align our roadmaps.

Asking the right questions at the right time

When meeting with the PMs of other teams or conducting user interviews, I needed to efficiently use my time and theirs to gather the most relevant information I could.

Prioritizing

There are a multitude of factors that go into prioritization - the feature’s impact, readiness, dependencies, alignment with our vision, time to build, and feasibility. I learned to balance and prioritize features on a case by case basis.

Process

1. Compile a list of teams you need to connect with

Working with the PM of AppsManger and researching internally, we were able to create a list of PAS and PAS adjacent teams that could have potential integration points with apps manager. We then tried to find their roadmaps or charters to see if collaboration with them lied in the near future (now - 3 months).

2. Set up meetings with relevant product managers

After looking at the roadmaps of different teams, we narrowed down a list of what teams we needed more information on. We prepared ourselves, knowing what info we needed and what questions to ask, and scheduled a follow up for next steps if needed.

3. Compile features from all teams

Once we gathered all the information we needed, we created a list of all the potential integration points/features that we’ve confirmed with other teams. We also gathered additional information on each feature, such as the # of customers impacted, dependencies, its timeline, customer validation, build time, and its priority on the other team.

4. Breaking down priorities

We created two 2x2s to narrow down which features to work on first. We compared impact to readiness, then, we looked at value and cost. To determine value, I looked at the customer base (who benefits and what percentage of customers are they) and impact (why is this feature important and how does it help developers achieve something valuable). For cost, I looked how soon we could start working on it (has the other team released this feature GA, alpha, or beta?) and asked our lead engineer to give a “t-shirt size” (S, M, L) of how much effort it would take to build. Finally, I laid out a ranked feature list so I could start working.

5. Execution

With priority determined, I began taking steps to understand, design, and validate each feature.

Scoping
- Confirm understanding on both teams, go through workflow and the end result of the feature
- Make sure everything is doable and agree on a timeline
Design studio
- White boarding on my own or with my team and the other team
Figure out initial design
- Purposefully choose the simplest, most functional workflow
- Mock MVP up in Figma and create a prototype
Test it with users
- Create an interview script
- Determine the goals and the outcomes desired from each interview
- Synthesize feedback and determine what needs to be changed in the design
Create stories
- Work with anchor on fleshing out the technicalities
- Create and point (determine difficulty of) stories
- Prioritize stories in backlog

Spring Cloud Services Integration

Goal: Making Pivotal Cloud Foundry (PCF) more uniform - the go-to platform for developers to have easy access to everything they need in one place

Spring and PCF are separate entities within Pivotal and we worked on bridging them together in apps manager through 3 new feature integrations:

1. Config server mirror sync

2. Managing secrets through the config server

3. Circuit breaker dashboard for applications

Designs

These are features that currently exist, whose functionality can be executed through the command line interface or external dashboards. I wanted to make these features easily accessible, discoverable, and usable for developers in the same place. Even though each feature was different, they were still part of the same integration and I had to think about them together.

Config Server Sync Mirrors

This feature enables the user to synchronize their config server with the mirror service for their git repo.

One button to send request allows for a simple, straight forward experience

Distinct "Config" tab enables discoverability

Managing Secrets

Allows users to manage the secrets on each of their applications bound to the spring cloud services config server service instance

Discoverability either through a separate tab in the service instance page or in the application page

Table display with row removal

Fly out to add relevant info

enable the user to:

1. see all the information at one glance

2. perform all actions from the same page

Circuit Breaker Visualization

Users can view the open and closed circuits pertaining to their application

Circuit Breaker Overview Page (max 3).pn

Separate tab in application page: there can be a lot of circuit breakers associated with one application
Warning sign within overview page: allows the user to know there is something broken in their application
Figuring out the most important metrics to feature on the dashboard: give developers a good overview
Designing the new visualization: make it readable and organized

User Interviews

We needed to interview people who were familiar with spring cloud services because the developers who would be using the service instance already use the config server. We wanted to figure out if we captured all the functionalities as before, but with a better flow. We asked for volunteers from the Spring Cloud Services team and interviewed a technical writer (writes documentation for SCS products) and the anchor. In addition, we interviewed a solutions architect on the app transformation team who was familiar with SCS and appsmanager.

After sitting in on a few user interviews, I had an idea of how to prepare and properly guide the user through the demo.

I first wrote out an introduction to the integration and an initial set of screening questions. Then for each demo (Config server mirror sync, managing secrets, and circuit breaker visualization) I wrote out:

my 3 goals/outcomes I wanted from the interview
the function of the feature
the context of the demo
the task I want the user to complete
potential questions to ask during the user flow
closing questions to summarize the user’s experience and pain points

Insights

In addition to new insights on the features, there are a few key things I learned from this process.

Force to user to talk out their thoughts for as long as possible
Make every question open-ended
Be a neutral toned active listener
Just because they like it, doesn’t mean it works or that the user understands it

Config Mirror Sync

Overloaded terms confused the user - config, configuration, update, and sync
There was visually connected information that was, in actuality, not related (synchronizing mirrors does not change the displayed configuration)
Users experienced uncertainty of completion for the status of the task

Managing Secrets

Users only navigated to the service instance page - they actively resisted navigating to an application
There’s no way to bulk remove or add secrets
UI is a very simple and straightforward flow that prevents developers from making mistakes in adding secrets, and it is easier than the current method

Circuit Breakers

Users wanted more info about an open circuit
There needs to be clearer indication of circuits' status on the overview page
The dashboard is a good snapshot and visualization of overall health of the circuit breakers
AppsManager enables an easier workflow to get to circuit breakers than before

Redesigns

After synthesizing the new problems that arose in the demos, we came up with potential fixes and decide which were necessary and in scope of the MVP.

Config Server Sync Mirrors

We actually told the users to update their config server instead of mirroring it. This caused confusion in what the task was and where to navigate to.
- We reworded the task in our interview script and provided a robust description of the action within the config tab
Our flash message indicated that the request to sync mirrors was sent, but the user wouldn’t know when their config server was successfully mirrored.
- We turned the flash message into an info flash, because the user would not be able to find the status of their sync within apps manager (that information is unavailable in the Config Server endpoint)

Managing Secrets

No one navigated to the application page, though users acknowledged that it was a possibility someone could think to do that
- Decided there was no harm in leaving that workflow available in the demo and that it needed further testing
Bulk remove
- Conducted research and found out that most users don’t have more than 10 secrets on their config server, so bulk remove is unnecessary and out of scope for the MVP
- Found out there was a way to add the same secret to all applications with special word "application," so we decided to inform users of the functionality with "all apps" in parentheses.
Not all users knew that changes would only take affect after the apps is restarting
- Added the info in the flash message so the user can complete adding a secret right away.

Circuit Breaker Visualization

Users wanted more information
- Creating a visualization that could hold more info and testing the best way to implement this change
  - Hover for new visualization
  - Click for new visualization
  - Change all the visualizations
  - Figure out relevant information developers need in each circuit state (open vs. closed)

Clearer warning of open circuits
- Changing the title to “open circuit breakers” and indicating how many there are

Project Learnings

Keep questioning

When interviewing a user, prioritizing, or designing, it’s important to keep asking why. You need to find the core reasoning for people’s actions and motivations in order to solve the real problem.

Include diverse skillsets

Remember to communicate with engineers, product managers, customers, and other teams throughout the process. That way the product maintains feasibility, business value, and stakeholder value.

Maintain high-level understanding

Instead of getting caught up in the technicalities and the details, you can focus on finding problems to solve and aligning your product with your vision and overall business objectives.