Design of Experiments for science workflows and pipelines

Dr Mark Kiemele1

1Air Academy Associates, 2295 Oracle Blvd.  Suite #340 Colorado Springs CO 80921, USA



It is not widely known that the manner in which data is collected highly influences how easy or how hard it will be to analyze and evaluate the results once the data has been collected. This workshop will address the fundamental principles of optimal data collection with regard to test and evaluation, with special emphasis on the ability to characterize the process or system under test after the data has been collected.
We will cover the major pillars for developing a culture of proper experimentation (aka Design of Experiments) and demonstrate each of the principles using a miniature roman catapult. The principles covered will include a technique for reducing the impact of extraneous variation and the use of measurement system analysis for the purpose of ascertaining the quality of data from a statistical perspective as well as conducting uncertainty analysis of the measurements. The statistical analysis of experiments will be shown to be impacted by the principles of orthogonality, replication, randomization, and blocking. These techniques and principles are not a substitute for prior process knowledge, engineering knowledge, or experience. But without applying them, engineers, researchers, scientists, or practitioners will not be as effective or as efficient as if they were to use these powerful and proven, but still largely unknown, tools. This tutorial does not require any prerequisite knowledge and will be beneficial to anyone who is associated with test and evaluation activities or would like to know more about the Design of Experiments methodology which encompasses all of the above topics.


Mark J. Kiemele, President and Co-founder of Air Academy Associates, has more than 30 years of teaching and consulting experience. Having trained, consulted, and mentored more than 30,000 leaders, scientists, engineers, managers, trainers, practitioners, and college students from more than 20 countries, he is world-renowned for his Knowledge Based KISS (Keep It Simple Statistically) approach to teaching statistical methods to practitioners. His support is requested by an impressive list of global clients, including Sony, Microsoft, GE, Raytheon, Lockheed-Martin, Northrop-Grumman, General Dynamics, GlaxoSmithKline, Samsung, PerkinElmer, Danaher, Corning, BASF, Dell/EMC, John Deere, Xerox, and a host of DoD organizations.

Mark earned a B.S. and M.S. in Mathematics from North Dakota State University and a Ph.D. in Computer Science from Texas A&M University. During his time in the U.S. Air Force, Mark supported the design, development and testing of various weapon systems, including the Maverick and Cruise Missile systems. He performed the first aircraft/missile safe escape analysis on the Maverick, determining optimum escape maneuvers to minimize risk to the launch aircraft and pilot. On the Cruise Missile system, he was a software engineer responsible for the design and development of the terrain following guidance system. Mark was also a professor at the U.S. Air Force Academy.

In addition to many published technical papers, he has co-authored the books Basic Statistics: Tools for Continuous Improvement; Knowledge Based Management (KBM); Applied Modeling and Simulation: An Integrated Approach to Development and Operation; Network Modeling, Simulation, and Analysis; Lean Six Sigma: A Tools Guide and most recently Design for Six Sigma: The Tool Guide for Practitioners. He is the editor of the text Understanding Industrial Designed Experiments. He is the 2017 winner of the International Test & Evaluation Association’s prestigious Richard G. Cross Award, which recognizes outstanding accomplishments in test and evaluation education.


AeRO is the industry association focused on eResearch in Australasia. We play a critical coordination role for our members, who are actively transforming research via Information Technology. Organisations join AeRO to advance their own capabilities and services, to collaborate and to network with peers. AeRO believes researchers and the sector significantly benefit from greater communication, coordination and sharing among the increasingly different and evolving service providers.

Conference Managers

Please contact the team at Conference Design with any questions regarding the conference.
© 2019 Conference Design Pty Ltd