Example
Consider an application offering users encrypted and secure access to documents. It has a back-end database storing the encrypted documents that can be retrieved by a document server. This server will decrypt the documents using the key of the database and immediately encrypt the documents using the key of the requesting user so that the user receives the document encrypted with her own key. When more and more users request documents, the load will increase on the document server, and one may need to adapt the application by adding one or more document servers with a load balancer to distribute the incoming user requests onto the available servers. How many document servers are needed to keep the current number of users happy?
Background
The different software modules of the above application (database, document server, load balancer) are generally black boxes running unknown code and algorithms. The Digital Twin can observe only the communication among the pods. Noting the time it takes from a component receives a request packet until it emits a response or a further request can be logged by the application. Hence, for each incoming packet of a given type one can build a model of the computing delay of the algorithms executed by the component. This delay model can then be used in simulations to predict the performance of the component when multiple independent instances will be operating concurrently. For the example, how long the document server needs to produce a response with a user encrypted document depends on the complexity of the algorithm, the size of the document requested, and the load on the server. Observing the variability of many such requests, one can say something about how long a user would need to wait. Adding more document servers will reduce the waiting time, but by how much? Each parallel server would still need to execute the algorithm, but there will be less competition over its computing resources from different requests. Can these delay factors be isolated and quantified?
Research questions
- Does packet traces contain enough information to identify the deterministic and contextual parts of an observed component's delay distribution?
- Can one learn the behaviour of each component?
- Can one connect learned models of the components into an end-to-end model for the whole application topology?
- Is it possible to use the models to draw conclusions about the performance of the application in different configurations (e.g., number of document servers) or different contexts (e.g., number of application users)?
- What are the uncertainties of the predictions and how do they scale with the number of components needed for the application?
Methodology
- Create or evaluate one or more case studies: develop the application to study from scratch or use existing case studies, for example Google Cloud Platform microservices demo.
- Add application-specific monitoring to the case studies, using an existing observability framework or monitoring functionality.
- Develop a test runner that puts the system under load by simulating end user behavior, and extract application traces from the running application.
- Replay the recorded traces on an existing parameterized digital twin model. Use existing optimization libraries or develop an algorithm to calibrate the digital twin so its simulated delay timings agree with the real system.
- Design experiments: modify the digital twin to simulate variations in deployment (increase or decrease in container or virtual machine capacity, increasing or decreasing replica count of microservices, etc.).
- Develop a quality criteria and measure how well the digital twin's behavior results in a prognosis of the behaviour of the real system.
- Evaluate the digital twin's accuracy by modifying the real system in equivalent ways and comparing prognosis and measurements.
What you will learn
- Performance evaluation methodologies
- Software application monitoring techniques
- Digital Twin model representation and implementation
- Design of experiments and evaluation of experimental outcome
- System performance prediction techniques