Open Ended Questions for Senior Software Testing Professionals

Following are few of the open ended questions in context of software testing for senior software testing professionals especially test managers.  Senior professionals should be having at least minimum awareness in context of these questions.  The experts in their areas should be able to answer related questions with greater length though.

Requirements & Specifications

• How to develop requirements with ‘Use Cases’?
• How to create a good test specification?
• How to spot and fix requirement errors early?
• How to distinguish business requirements from system requirements?
• How to assess that the requirements are complete?
• How many tests do you suggest per requirement?
• What is the importance of testing based on business, as well as system, requirements?
• What is your opinion on developer’s inclusion in requirements-based unit tests?
• How to develop test cases when requirements are not available or documented only at very high level?
• How do you generate test requirements when it doesn’t exist?

Agile Methodology

• How to practice agile methodology pragmatically in real world adoption?
• What are different agile methods?
• How the SCRUM methodology applies to the testing effort?
• How to define a testing sprint goal with your key customers?
• How to manage testing as a product backlog activity, including defining the testing focus with the customers?
• What do you think the value of daily stand-up meeting in managing the testing cycle, and how to implement them correctly?
• Why a testing sprint review is important to set the stage for the next testing cycle?

Performance Testing

• How can you measure application performance without knowing requirements?
• How can you harness the cloud for performance testing?
• Why do you think performance should be managed by testing early and often?
• What do you really think affects web application performance?
• How can you develop effective performance test strategy?
• How to setup and maintain an effective performance test lab?
• How do you factor business and technical contexts in performance testing?
• How do you perform quick initial impact assessments to justify the performance testing?
• How do you facilitate tuning, debugging, fixing, capacity planning and right-sizing?
• How do you risk assessment to focus and prioritize the performance testing efforts?
• How do you test for scalability?
• How do you determine the loads to test with?
• How do you determine what tools, equipments and facilities to have in the test lab?
• How do you decide what to observe and where to monitor during testing?
• How to work with experienced performance testers to get the results you need, even if that can be verbalized yet?
• Why performance testing begin should begin well before the application is fully functional and how to do it?
• How to recognize the difference between “delivery” and “done” as they relate to performance testing?
• How to assess and balance the risks in performance testing?
• What can be different ways to better integrate performance testing personnel into the development team effort and vice versa?
• What can be performance issues across the entire application life cycle?
• How do you pinpoint and exploit specific performance bottlenecks?
• How can you extrapolate results from a test environment?
• How do you go for generating load in complex environment?
• How do you differentiate performance symptoms, caused and potential cures?
• What do you think in terms of producing non-actionable test results in context of performance testing?  Can you think of some instances?
• What are the pitfalls of using automated performance testing tools?
• What is the use of extrapolation in performance testing?
• What do you understand by software robustness? How do you verify software robustness?
• What are some common failures in application performance? How do understand them?
• How do you do performance management throughout the application life cycle?
• How do you setup a lab in context of performance testing that is ready to respond?

Security Testing

• How hackers use HTTP(S) as a tool to break into your sensitive networks?
• How to defend against Web application attacks?
• What are some of the most dangerous application security vulnerabilities and how to test for them?
• What are different data protection measures for multi tier architectures?
• What are some of the common strategies and methodologies used by hackers?
• What is SQL Injection and how to write test cases for it? How do you automate SQL injections test cases?
• What are buffer overflows and how a hacker can manipulate it?
• How to secure site data and user confidential information housed in DB server?
• What are dangers that components such as ActiveX controls and cookies can pose for exploits and loss of user privacy?
• How to protect transmission of critical user data, such as payment and other private information via secure protocols?
• What would be high level strategies for testing the security of a web site, which can be used as the basis for security test case development?

Test Automation

• What are top challenges of automation?
• Why do you think automation project fails? What are some of the potential problems and roadblocks for test automation?
• How do you put together the case for automation?
• How do you build yours own automation framework?
• How can you make yours automation scripts more robust?
• How do you ensure efficiency of GUI test automation?
• What is yours strategy of test automation for changing software?
• What are some major principals and current industry standards of GUI test automation?
• How do you decide if a specific project should be automated or not?
• How do you define a scope for test automation?
• How do go for selecting a test tool for automating a specific application?
• How do you build a team for test automation?
• How do you select a test automation framework that fits yours test automation needs?
• How do you manage test automation projects?
• Why testing of database objects and stored procedures are necessary and what problem you see in existing popular automation tool?
• How do set some practical goals for test automation?
• What are the strategies for global test automation?
• How do you know when and how to leverage automation technology to maximize speed?
• How do you implement smoke tests?
• What are the problems with using intrusive automated testing tools?
• Why software developers need to keep automated testing tools capabilities in mind?
• How do you analyze tool upgrades and how the new tool’s features will affect existing test cases or existing functionality?
• What are the real benefits of automated testing?
• How do you maintain automated unit and system tests?
• When do you think that automation testing doesn’t speed up the testing effort?
• How to avoid using various tools and maintaining duplicate information in various repositories?
• How to manage test automation across many locations?
• How to deliver test automation successfully through people, methods and tools?
• What are some different types of framework? How are they different and what are their advantages / disadvantages?
• How to use framework like action based testing to ensure visibility, maintainability and scalability?
• How to incorporate an automation framework along with yours existing process?
• How you will go for developing yours own test automation tool?
• How to manage the “silver-bullet” expectations surrounding automation testing?
• What are different ways of defining automation criteria?
• How to create reusable automated test cases?
• How do you verify all vendor claims in yours test environment?
• How do you go for selecting the right tool?
• How do you go for buy or built tool decision?

Testing Techniques

• What could be some better way of bug hunting?
• What is strength and weaknesses of requirement based tests?
• What techniques you use for testing flash application?
• What are agile and high speed software testing techniques?
• What techniques you use for testing AJAX application?
• What are the challenges in doing multi users testing?
• How can you test the SOA architecture?
• How can you maximize coverage with minimum testing?
• What could be test methodology for XML web services?
• How can you improve code coverage with pair-wise testing?
• How do you test XML?
• What do you understand by ‘Differential Testing’?
• How to adapt familiar testing paradigms for emerging technologies?
• What is action based testing?  What do you understand by Soap Opera testing?
• What do you understand by ‘Functional Testing’ and ‘Structural Testing’?
• What do you understand by ‘Exploratory Testing’, ‘Boundary Value Analysis’, ‘Equivalence Class Partitioning’ and ‘Combinatorial Analysis’?
• What do you understand by ‘Statement coverage’, ‘Decision / branch coverage’ and ‘Condition and basis path coverage’?
• What do you understand by ‘Testing Patterns’?
• What would be yours test strategy for web testing?
• What do you understand by rapid business driven testing?
• What do you know about model based testing for java and web based GUI applications?
• What are some of the worst testing practices?
• Why black box testing itself is inefficient but gray box testing is more efficient?
• What do you understand by risk-based testing techniques?

Team Management

• How do you recruit, Hire, motivate and retain top testing talent?
• How to optimize the use of yours testing staff’s diverse skill sets?
• How can you increase the effectiveness of your team with quality feedback?
• How do you assess your test team effectiveness, efficiencies and more?
• How to use effective methods for measuring improvements of yours team?
• How can you create and lead the high performance test organization?
• How can you efficiently train your offshore test team? How do you work with offshore test team?
• How do you manage yours global test team work effectively?
• How do you minimize the costs and risks of global resources?
• What are some of the best practices for managing distributed testing team?
• How can you use virtualization to manage globally distributed testing?
• How can you justify test team existence in a company?
• How do you go for building collaborative performance testing and tuning teams?
• How to setup the right team and how to gain commitment from other managers and stakeholders?
• How you will go for creating a culture for change and implement change?
• How a testing team can maintain independence even if the development and testing teams have the same reporting structure?
• What should be roles and responsibilities of integrated testing team members?
• How do you manage cultural shocks in case change of organizational structural change?
• How do you recognize warning signs that the finger of blame is moving in yours direction, and what you do in that case?
• How do you avoid finger of blame and bring development and testing together with business side?

Schedule Management

• How can you setup and track realistic schedules?
• What are few fatal flaws in estimating tests?
• What would be some tips and best practices that will help to keep the project on track?
• How do you predict testing, delivery dates and cost overruns?

Project Management

• How can you achieve productivity and quality with agile methods?
• What do you understand by ALM (Application Lifecycle Management) tool?
• What do you think is the foundation of good testing?
• What do you understand by TCoE (Test Center of Excellence)?
• What is yours strategy of picking best free software?
• How do you use SCRUM as test management?
• How do you determine the scope of a testing strategy for software projects? How do you decide what not to test?
• How do you manage ‘Acceptance’ testing cycles?
• How do you review software development for testability?
• How do you decide that yours product is ready to ship?
• How do you plan and managing a Beta Test Program?
• Why an understanding of system architecture and underlying components is necessary in order to develop effective gray box test cases?
• What could be practical strategies for today’s iterative development environment?
• How you finish the project?  What you do for the same?
• What techniques you apply to stop the cycle of “triage testing”?
• How to assess both the investment and the return in languages of business?
• How to do value modeling that will provide essential credibility for dollar figures?
• How to put dollar values on soft intangibles?
• How do you align testing with business processes and development practices?
• What are your ways to understand the perspective of the ultimate end user, and what it means for you?
• How do you manage expectations?
• How to extract better requirements and how to get full buy-in when you do?
• How do you deal with test tool vendors?
• How do you develop and deliver yours value proposition to business side?
• Do you bother about coding standard and unit testing? Why?
• How system testability can be increased?
• How do you manage and track software testing in chaotic environments – especially projects with continuously changing requirements and shifting priorities?
• How do you decide how much is yours software quality worth?
• How do you perform unit testing in agile methodology?
• How the development approach can be structured to support effective unit testing?
• How to apply quality assurance across the entire application development life cycle?

  

Metrics

• How do you design and utilize test metrics?
• How to use metrics to improve software testing?
• What are ways to provide metrics that really matter to organizations and provide visibility into their or customer’s organizations?
• Why metrics program fails?
• How do you understand yours statistics for different projects before collecting them?
• What are keys to successful metrics programs?
• How to develop quality metrics that matter and ways to implement and maintain these metrics over time?
• How do you measure, analyze and optimize for continuous improvements?
• What kind of code metrics do you use for targeted code re factoring?
• What are different code coverage metrics you are aware of?

Testing Processes

• How can you build a quality driven development process?
• What do you think should be best defect workflow?
• How do you go for software process improvement methods that are low overhead?
• How do you practice failure mode analysis process? What are its benefits and pit falls?
• How can you build a culture in organization that views failures as process and not just a single event?
• How can you sell fault analysis effort to manage as a means to improve the quality, reliability and maintainability of software systems?
• What is the appropriate level of test case detail do you suggests / like in projects?
• What do you think is role of test / QA?
• How do you conduct code review?
• What practices should you employ if yours software can never be allowed to fail?
• How to evaluate yours test process and strategy?

  

Defects Management

• How do you manage bugs?
• How do you root cause analysis of the defects?
• How do you decide which bugs to fix and which ones to keep?  How do you take these decisions consistently?
• How to prioritize defects in order to meet the go-live date?
• What do you understand by S-curves and zero bug bounce?
• How can you bring down quality cost through defect management?