Search This Blog


Thursday, July 10, 2014

ASP.NET Application - What to Measure for Performance?

ASP.NET Application – What to Measure for Performance?
»        What are number of requests getting executed per second?
»        How many numbers of requests waiting to be executed?
»        How many numbers of requests being rejected?
ASP.NET Applications \ Requests / sec
Web Service \ ISAPI Extension Requests / sec
ASP.NET \ Requests Current
ASP.NET Applications \ Requests Executing
ASP.NET Applications \ Requests Timed Out
Cost of Throughput
»        What is the cost of throughput in term of processor utilization, memory, disk I/O and network utilization?
»        What is the cost of hardware needed to achieve given level of performance?
Processor \ % Processor Time
Processor \ % Privileged Time
Processor \ % Interrupt Time
System \ Processor Queue Length
System \ Context Switches / sec

Memory \ Available Mbytes
Memory \ Page Reads / sec
Memory \ Pages / sec
Memory \ Pool Nonpaged Bytes
Server \ Pool Nonpaged Failures
Server \ Pool Paged Failures
Server \ Pool Nonpaged Peak
Memory \ Cache Bytes
Memory \ Cache Faults / sec
Cache \ MDL Reads Hits %

Disk I/O
Physical Disk \ Average Disk Queue Length
Physical Disk \ Avg. Disk Read Queue Length
Physical Disk \ Avg. Disk Write Queue Length
Physical Disk \ Avg. Disk sec / Read
Physical Disk \ Avg. Disk sec / Transfer
Physical Disk \ Disk Writes / sec

Network I/O
Network Interface \ Bytes  Total / sec
Network Interface \ Bytes Received / sec
Network Interface \ Bytes Sent / sec
Server \ Bytes Total / sec
\ Segment Received / sec
\ Segment Sent / sec
Processor \ % Interrupt Time
»        What numbers of requests queued in IIS?
»        What are number of requests in application queue?
»        How many requests got rejected because request queue was full?
»        What is average wait time for requests waiting in queue?
ASP.NET \ Requests Queued
ASP.NET Applications \ Requests in Application Queue
ASP.NET \ Requests Rejected
ASP.NET \ Requests Wait Time
Response Time & Latency
»        What is the time for TTFB (Time to First Byte?
»        What is the time for TTLB (Time to Last Byte)
»        What is processing time of ASP.NET requests?
TTFB (from client side)
TTLB (from client side)
ASP.NET \ Request Execution Time
Cache Utilization
»        What is current number of entries (users and internal) in the cache?
»        How the cache utilization look like?
»        Are there lots of updates happening in cache?
»        How is cache being utilized from user code?
»        Are high numbers of items being added and removed from user cache?
»        What are number of entries in output cache?
»        How is cache being utilized in output cache?
»        Are high numbers of items being added and removed from output cache?
ASP.NET Applications\Cache Total Entries
ASP.NET Applications\Cache Total Hit Ratio
ASP.NET Applications\Cache Total Turnover Rate
ASP.NET Applications\Cache API Hit Ratio
ASP.NET Applications\Cache API Turnover Rate
ASP.NET Applications\Output Cache Entries
ASP.NET Applications\Output Cache Hit Ratio
ASP.NET Applications\Output Cache Hit Ratio
ASP.NET Applications\Output Cache Turnover Rate
Errors & Exceptions
»        How many exceptions were generated during preprocessing, parsing, compilation and run-time processing of requests?
»        How many errors occurred during processing of requests?
»        What is total number of unhandled exceptions occurred per second during run time?
ASP.NET Applications\ Errors Total/sec
ASP.NET Applications\ Errors During Execution
ASP.NET Applications\ Errors Unhandled During Execution/sec
»        What is session timeout value? Is this optimum?
»        How much server resources being consumed by sessions?
ASP.NET Applications \ Requests / Sec
Processor \ % Processor Time
Memory \ Available Bytes
Process \ Private Bytes
»        How many number of assemblies and application domains loaded?
»        What is the amount of committed memory across all application domains?
.NET CLR Loading\ Current appdomains
.NET CLR Loading\ Current Assemblies
.NET CLR Loading\ Bytes in Loader Heap
View State Size
»        What is size of view state across all pages?
Enable trace log
Page Size
»        What are sizes of response sent by different pages?
Enable logging of bytes sent in IIS
Page Cost
»        What is the page cost in terms of CPU cycles?
»        What is the page cost in terms of time?
Cost (Mcycles/request) = ((number of processors x processor speed) x processor use))/ Number of requests per second
Worker Process Restarts
»        How many times the web application recycles and the worker process recycle?

ASP.NET\ Worker Process Restarts

ASP.NET Application Performance Considerations

ASP.NET Application Performance Considerations
Threading Efficiency
Thread pool tuning, Burst load, Creation of thread on per request, Blocking threads, Asynchronous calls
Pool Resources
Same connection string, Dispose or Close Call, Caching of pool resources, Obtain resource and release time
Page timeout, Page Size, Buffering, Page.IsPostBack, Partition Page Content, Expensive loops, Sever.Transfer instead of Response.Redirect
Server Controls
View State in Server Control, Deep hierarchies of Server Control
Data Binding
Page.DataBind, DataBinder.Eval
Dynamic data and static data, Memory limit, Caching right data, Refreshing cache, Cache API, Output Caching, Partial Page or Fragment Caching, Cache location, Kernel caching
State Management
InProc, StateServer,  SQL Server
View State
Disabling of View State, Number of objects in View State, Size of View State
String Management
Response.Write for formatting output, String builder for temporary buffers, HTMLTextWriter for custom controls, Large number of temporary allocations in short time
Exception Management
Global error handler, Number of exceptions thrown, Try/Finally on disposable resources, Code to avoid exceptions,
Data Access
Paging for large result set, DataReader vs DataSet, Requesting too much data
Unwanted web server traffic, Authentication for anonymous access, Validation on client-side, Per-request impersonation, Caching sensitive data, Secure & non-secure content, SSL for selective pages, Absolute URLs for Navigation between HTTP and HTTPs, SSL Hardware, SSL Timeout
Overhead of inter process and inter computer communication, Memory limit configuration, Tracing & Debugging, Web garden configuration, HTTP Compression

Thursday, June 26, 2014

Usage of Mind Map in Performance Engineering

Mind maps can be fantastic tools to aid analyzing performance bottlenecks.  If done correctly, it not only help identifying bottlenecks faster but document analysis tips for rest of the team and for later usage.  Below mind map is an example of root cause analysis when page load time is observed to be high.

Most of the time, when response time of a web request is found to be high, performance tester starts digging out the fault on server side.  It is not always true to have the issue lying at server side and on database server.  The issue can exist anywhere in the delivery chain.  I usually follow below order while identifying the issue of high response time.

Step 1:  Issue with Tool / Scripts
Before checking anywhere else, it is advisable to validate the scripts and tools being used for performance testing.
»        Does high response time include think time unknowingly?
»        Is there redundant logic (e.g. client side functionality) inside transaction for which response time is reported?
»        Which tool is being used for load generation? What is the logic for calculating response time?
»        Is there any extended logging being used?  Extended logging will skew the response time.
»        Has browser (number of parallel connections) emulated realistically?

Step 2:  Issue with Infrastructure where load generators are installed
It is important to monitor hardware performance where load generators are installed.  The response time information might be incorrectly stated when low configuration machines are used.
»        Are hardware utilization (CPU, Memory, I/O) overly stressed?
»        Are there issues (Latency, Packet Loss and Bandwidth) with network?

Step 3:  Issue with client side logic and web page resources
Response time can be improved by optimizing the client side logic first before tweaking the server side logic
»        Are images being scaled unnecessarily? Avoid scaling of images in HTML reduces the bandwidth usage and round trip time
»        Is CDN being used for serving the static resources? CDN helps in serving the static resources faster
»        Is JS / CSS blocking loading of other resources? Loading of JS / CSS can be deferred to unblock other resources
»        Is unused code also being downloaded? Apart from performance, it risks security as well
»        Are there many third party ads being loaded?  Is third party ad causing slow response time?
»        Is caching being used properly? Caching helps in serving the requests faster.
»        Is cookie less domain being used for static resources? It helps in optimizing the bandwidth.
»        Are images being loaded in parallel? How many parallel threads being used?
»        Are images optimized for better serving and bandwidth?
»        Is CSS sprite being used for loading logos?
»        Are resources being compressed before serving?
»        Has redundant code and space being removed before serving JS / CSS files?
»        Are view states being used properly?
»        Is this possible to reduce number of requests by validating content on client side rather than on server side?

Step 4:  Issue with load balancer
»        Is load balancer algorithm working as expected? Are all requests being distributed evenly on active servers?

Step 5:  Issue with Web Server
»        Does it require improving hardware (CPU, Memory, I/O) of web server?
»        Is caching being utilized properly on web server?
»        How threads are being utilized? Are there enough threads allocated in thread pool?
»        Are there requests waiting for external resources (database, disk, payment gateway etc.)?
»        Is web server configured as per required load? Configuration related to thread pool, number of concurrent users allowed etc.?

Step 6:  Issue with Application Server
»        Does it require improving hardware (CPU, Memory, I/O) of application server?
»        Is JVM configured properly for heap size, thread models, GC scheme and JVM version?
»        Is garbage collector working efficiently? Can it be optimized further by sizing the generation and selecting a proper collector?
»        Is caching being utilized properly on application server? Connection cache? File cache?
»        Is thread pool configured properly? Maximum threads? Queue length?
»        Are sessions being configured properly like session time out?
»        Are many exceptions stack trace found in debug log file?
»        Are asynchronous calls being handled optimistically?

Step 7:  Issue with Database Server
»        Does it require improving hardware (CPU, Memory, I/O) of database server?
»        Are indexes being used properly?
»        Are there expensive queries that can be tweaked / improved?
»        Are there blocking issues due to locks?
»        Are indexes heavily fragmented?  Is database in need of defragmentation?
»        Is it a replication intensive environment? Is replication is causing slowness?
»        Has database statistics been updated?  When was the last time database statistics run?