This paper is full of wisdom. A selection of quotes that are as pertinent today as they were 40 years ago.
“When I pointed this out to engineers their response then was people should just pay attention to all alarms regardless. When I heard developers say people should just be more careful, I knew that statement was an indicator of the design of a bad human-machine system. “
Mastodon Source 🐘
“Meanwhile it is still popular to build either/or architectures where the machine does all the work and people are supposed to monitor the machine to jump in when it is doing the wrong thing. Or…the machine does all the work till it can no longer, and then it dumps the messy situation into some human’s hands—bumpy transfers of control. With every either/or architecture comes new opportunities to observe the same unintended and undesirable effects as I observed in the mid-1980’s”
Mastodon Source 🐘
“By the way, when it comes to all things AI, remember there is always a hidden trick — good CSE-ers find ways to break automata and plans by focusing on patterns of demands that challenge any agent or set of agents whatever the combination of human and machine roles.“
Mastodon Source 🐘
D Woods, David. 2016. “Origins of Cognitive Systems Engineering.” 2016. https://www.researchgate.net/publication/298793082_Origins_of_Cognitive_Systems_Engineering.