── Michael V. Hayden: when we went to them for things nobody had done yet, we found that at best they weren't much better or faster than we were.
── And that was true even with a team that included such defense giants as SAIC, Boeing, CSC, AT&T, and Booz Allen Hamilton. We were also trying to do too much, too quickly.
── We would have been better advised to pick our spots and work incrementally, trusting to spiral development to eventually get us to where we wanted to be.
── (Michael V. Hayden, Playing to the edge : American intelligence in the age of terror, 2016, pp.20-21)
Michael V. Hayden, Playing to the edge : American intelligence in the age of terror, 2016
p.20
We found that when we went to industry for things they already knew how to do, we got impressive results. When we went to them for things nobody had done yet, we found that at best they weren't much better or faster than we were. And that was true even with a team that included such defense giants as SAIC, Boeing, CSC, AT&T, and Booz Allen Hamilton.
We were also trying to do too much, too quickly. Trailblazer comprised multiple moon shots.
([ DARPA - learning curve ])
p.21
We would have been better advised to pick our spots and work incrementally, trusting to spiral development to eventually get us to where we wanted to be.
(Playing to the edge : American intelligence in the age of terror / Michael V. Hayden, New York : Penguin Press, 2016, (hardback) (ebook), intelligence service──united states. | national security──united states. | united states. central intelligence agency. | united states. national security agency. | biography & autobiography / political. | political science / political freedom & security / intelligence. | history / united states / 21st century., JK468.I6 H39 2016 (print), JK468.I6 (ebook), 327.1273──dc23, 2016, )
[[ manufacturing ]]
[[ prototype ]]
[[ how many prototype did you build before you are satisfy with product ]]
____________________________________
Ashton Carter, Inside the five-sided box : lessons from a lifetime of leadership in the Pentagon, [2019]
p.324
The biggest challenge was connecting DARPA's breakthroughs to real military hardware. The “valley of death” that separates R&D from production is an age-old problem in technology, and bridging it is a vital challenge that every senior tech manager must tackle.
p.325
that divides research and engineering from acquisition in two separate organizations!
You might as well write a statute repealing the laws of thermodynamics (which explain why a perpetual motion machine is impossible).
( Inside the five-sided box: lessons from a lifetime of leadership in the Pentagon / Ash Carter.; Ashton B. Carter, author.; [2019]; subjects: Carter, Ashton B. | united states. department of defense. | united states ──military policy ── decision making. | pentagon (va.) | united stats. department of defense ── history. | united states. department of defense ── procurement. | united states. department of defense ── officials and employees ── biography. | cabinet officers ── united states ── biography.; UA23.C2746 2019; DDC 355.6092 [B]──dc23 )
____________________________________
Jon Gernter., The idea factory : the Bell Labs and the great age of American innovation, [2012]
p.22
Then Jewett asked his friend for help. “Let us have one or two, or even three, of the best of the young men who are taking their doctorates with you and are intimately familiar with your field. Let us take them into our laboratory in New York and assign to them the sole task of developing a telephone repeater.”18
Here was a new approach to solving an industrial problem, an approach that looked not to engineers but to scientists.
p.23
Within two years Arnold came up with several possible solution to the repeater problem, but he mainly went to work on improving an amplifier known as the audion that had been brought to AT&T in 1912 by an independent, Yale-trained inventor named Lee De Forest. The early audion was vaguely magical. It resembled a small incandescent light bulb, yet instead of a hot wire filament strung between two supporting wires it had three elements ── a metal filament that would get hot and emit electrons (called a cathode); a metal plate that would stay cool and attract electrons (called an anode); and between them a wire mesh, or “grid”. A small electrical current, or signal, that was applied to the audion's grid could be greatly amplified by another electrical current that was traveling from the hot cathode to the cool anode. Arnold found, through trial and error, the best materials, as well as a superior way to evacuate the air inside the audion tube. (He suspected correctly that a high vacuum would greatly improve the audion's efficiency.) Once Arnold had refined the audion, he, Jewett, and Millikan convened in Philadelphia to test it against other potential repeater ideas. The men listened in on phone conversations that were passed through the various repeaters, and they found the audion clearly superior. Soon to be known as the vacuum tube, it and its descendants would revolutionize 20th century communications.
p.32
That an industrial laboratory would focus on research and development was not entirely novel; a few large German chemical and pharmaceutical companies had tried it successfully a half century before. But Bell Labs seemed to have embraced the idea on an entirely different scale.
p.32
Harold Arnold
As Arnold explained, his department would include, “the fields of physical and organic chemistry, of metallurgy, of magnetism, of electrical conduction, of radiation, of electronics, of acoustics, of phonetics, of optics, of mathematics, of mechanics, and even of physiology, of psychology, and of meteorology.”10
p.51
In truth, large leaps forward in technology rarely have a precise point of origin. At the start, forces that precede an invention merely begin to align, often imperceptibly, as a group of people and ideas converge, until over the course of months or years (or decades) they gain clarity and momentum and the help of additional ideas and actors. Luck seems to matter, and so does timing, for it tends to be the case that the right answers, the right people, the right place ── perhaps all three ── require a serendipitous encounter with the right problem. And then ── sometimes ── a leap. Only in retrospect do such leaps look obvious. When Niels Bohr ── along with Einstein, the world's greatest physicist ── heard in 1938 that splitting a uranium atom could yield a tremendous burst of energy, he splapped his head and said, “Oh, what idiots we have been!”11
p.113
There had been whispers in the electronics industry about whether Bell Labs' enthusiasm over the transistor was overblown; the reported difficulty in manufacturing the devices only added to the skepticism. Whether it was a shortcoming or an advantage, Kelly's confidence was almost certainly rooted in his early experiences. He remembered the endless days and nights constructing vacuum tubes in lower Manhattan, the countless problems in the beginning and then the stream of incremental developments that improved the tubes' performance and durability to once-unimaginable levels. He could remember, too, that as the tubes became increasingly common ── in the phone system, radios, televisions, automobiles, and the like ── they had come down to price levels that once seemed impossible. He had long understood that innovation was a matter of economic imperatives. As Jack Morton had said, if you hadn't sold anything you hadn't innovated, and without an affordable price you could never sell anything. So Kelly looked at the transistor and saw the past, and the past was tubes. He thereby intuited the future.
pp.149─150
Mervin Kelly
March 23, 1950
a polished version of the lecture about Bell Labs
Royal society
Bell Labs was the world's foremost example of a place where scientists pursued creative technology.
p.150
Bell Labs helped maintain and improve that system, he said, by creating an organization that could be divided into three groups. The first group was research, where scientists and engineers provided “the reservior of completely new knowledge, principles, materials, methods and art.” The second group was in systems engineering, a discipline started by the Labs, where engineers kept one eye on the reservoir of new knowledge and another on the existing phone system and analyzed how to integrate the two. In other words, the systems engineers considered whether new applications were possible, plausible, necessary, and economical. That's when the third group came in. These were the engineers who developed and designed new devices, switches, and transmissions systems. In Kelly's sketch, ideas usually moved from (1) discovery, to (2) development, to (3) manufacture.
p.150
the telephone system, Mervin Kelly
p.151
a living organism
In truth, the handoff between the three departments at Bell Labs was often (and intentionally) quite casual. Part of what seemed to make the Labs “a living organism,” Kelly explained, were social and professional exchanges that moved back and forth, in all directions, between the pure researchers on one side and the applied engineers on the other. These were formal talks and informal chats, and they were always encouraged, both as a matter of policy and by the inventive design of the Murray Hill building. Researchers and engineers would find themselves discussing their respective problems in the halls, over lunch, or they might be paired together on a project, either at their own request or by managers. Or a staffer with a question would casually seek out an expert, “whether he be a mathematician, a metallurgist, an organic chemist, an electromagnetic propagation physicist, or an electron device specialist.”
p.151
Physical proximity, in Kelly's view, was everything. People had to be near one another. Phone calls alone wouldn't do.
Kelly had even gone so far as to create “branch laboratories” as Western Electric factories so that Bell Labs scientists could get more closely involved in the transition of their work from development to manufacture.
p.152
What he went on to describe in London, though, was a systematized approach to innovation, the fruit of three decades of consideration at the Labs. To Kelly, inventing the future wasn't just a matter of inventing things for the future; it also entailed inventing ways to invent those things.
Bell Labs' experience over the past few years demonstrated that the process of innovation could now be professionally fostered and managed with a large degree of success ── and even, perhaps, with predict ability.
Industrial science was now working on a scale, and embracing a complexity,
p.152
He had a formula
p.152
In TECHNOLOGY, the odds of making something truly new and popular have always tilted toward failure.
(The idea factory : the Bell Labs and the great age of American innovation / Jon Gernter.
1. bell telephone laboratories──history──20th century.
2. telecommunication──united states──history──20th century.
3. technological innovations──united states──history──20th century.
4. creative ability──united states──history──20th century.
5. inventors──united states──history──20th century.
TK5102.3.U6G47 2012
384──dc23
384 Gernter
)
____________________________________
National academy of sciences
A biographical memoir
of
Mervin Joe Kelly (1894─1971)
by John R. Pierce
copyright 1975
Mervin Joe Kelly (February 14, 1894─March 18, 1971)
creative technical management.
As Frederick R. Kappel, former board chairman of AT&T said after
Kelly's death:
"He was a great fellow for the Bell System. Mervin was
always and forever pushing the operating management, and the
heads of AT&T as well, to get on with new things. His aggres-
siveness got him in a lot of hot arguments, but I always sat back
and said, 'Give it to them, Mervin, that's what we need.' Every
place needs a fireball or spark plug, and he was it."
Others were less disturbed by Kelly's temper. Estill Green
describes his experience as vice president in charge of systems
engineering in these mellow words:
"A few years in close association with Mervin were the hap-
piest time of my life. For years on end I had believed I needed
insulation from the high voltage. Yet when I was directly ex-
posed to it, I never experienced a serious shock, and I rejoiced
to observe how the high potential overpowered inertia and loose
thinking and prejudice.
"I learned never to oppose him when he had the bit in his
teeth. Next morning I could remark casually, 'Mervin, there
are some aspects of that matter discussed in yesterday's confer-
ence that you may not be fully aware of.' He would listen, and
generally modify his position, to a minor or sometimes major
extent."
In 1943 Kelly outlined a branch-laboratory concept. This
eventually led to the establishment of laboratories for final
development at manufacturing locations of Western Electric.
This proved important in several ways. It linked final develop-
ment and its procedures and personnel closely to those respon-
sible for the manufacture of new devices and systems. It pre-
vented too large a concentration of personnel in a few central
locations. It gave a desirable measure of responsibility and
independence to work in various well-defined fields of devel-
opment.
'organized creative technology'
While Kelly recognized basic research as the source of all
technological advances, he understood that a complicated tech-
nological process lies between discovery and use. He wrote:
"There has been so much emphasis on industrial research
and mass-production methods in my country, that even our
well-informed public is not sufficiently aware of the necessary
and most important chain of events that lies between the initial
step of basic research and the terminal operation of manufac-
ture. In order to stress the continuity of procedures from re-
search to engineering of product into manufacture and to
emphasize their real unity, I speak of them as the single entity
'organized creative technology'."
Using the Bell Laboratories as an example of organized tech-
nology, Kelly delineated three areas that preceded the manu-
facture of complicated technological systems:
"The first includes all of the research and fundamental
development. This is our non-scheduled area of work. It pro-
vides the reservoir of completely new knowledge, principles,
materials, methods, and art that are essential for the develop-
ment of new communications systems and facilities.
"The second we call 'systems engineering'. Its major respon-
sibility is the determination of the new specific systems and
facilities development projects—their operational and economic
objectives and the broad technical plan to be followed. 'Systems
engineering' controls and guides the use of the new knowledge
obtained from the research and fundamental development pro-
grams in the creation of new telephone services and the im-
provement and lowering of cost of services already established.
"The third encompasses all specific development and design
of new systems and facilities. The work is most carefully pro-
grammed in conformity with the plan established by the systems
engineering studies. Our research and fundamental develop-
ment programs supply the new knowledge required in meeting
the objectives of the new specific devleopments."
Concerning Systems Engineering, Kelly said:
"Approximately 10% of our scientific and technical staff are
allotted to systems engineering. Its staff members must supply
a proper blending of competence and background in each of
the three areas that it contacts: research and fundamental devel-
opment, specific systems and facilities development, and opera-
tions. It is therefore, largely made up of men drawn from these
areas who have exhibited unusual talents in analysis and the
objectivity so essential to their appraisal responsibility."
Kelly illustrated an ideal relation between systems engi-
neering, research and development by the case of the NIKE
antiaircraft missile:
"For example, the programming study on the NIKE missile
system established that basic knowledge and art were available
for the development of a system that would meet the service
requirements except for a particular area of radar technology.
This area was at once subjected to a research and exploratory
development attack. The project was not undertaken until this
deficiency was eliminated by new knowledge from research.
The NIKE missile system now in production meets the require-
ments initially agreed upon and in its technical character is in
close correspondence with the plan of the initial study.
"I am familiar with large military systems developments
where this approach is absent, where research and exploration
are intermingled with specific development, probably with the
intent of gaining time. Actually, time has been lost."
Kelly goes on to say that development, while a continuous
operation, is done in three distinct stages: first the laboratory
model; after tests and modifications, the preproduction model,
which is field tested; and finally, the final design for manufac-
ture (by Western Electric).
Above all, a technological organization must have the lead-
ership to see and pursue real opportunities and real needs. In
an address to a naval research conference, Kelly said:
"The first, and perhaps the most important, factor is the
program itself. What shall it contain? What can be discarded
at once, and what shall be eliminated after limited exploration?
How can comprehensive coverage with freedom from gaps be
assured? In an endeavor so broad in scope and requiring such
a highly functional organization for its operation, how can
unneeded duplication be prevented, and duplication that is
worthwhile, though usually small in volume, be provided?"
Leaders or
managers must be technologically trained and technologically
competent. Only thus can decisions be based on insight and
understanding rather than on salesmanship and hearsay.
source:
National academy of sciences
A biographical memoir
of
Mervin Joe Kelly (1894─1971)
by John R. Pierce
copyright 1975
Mervin Joe Kelly (February 14, 1894─March 18, 1971)
____________________________________
[ new knowledge ]
“About new knowledge two points are clear. It cannot be forecast; and it cannot be achieved through administrative action. All that can be done is to create optimal conditions for its production.”
The Rockefeller university achievements 1901-2001
The Rockefeller university achievements
a century of science for the benefit of humankind
1901-2001
by Elizabeth Hanson
introduction by Arnold J. Levine
epilogue by David Rockefeller
2000
p.75
Herbert Spencer Gasser
electrophysiological techniques for studying nerves.
studies into the nature of nerve conduction, carried out with Joseph Erlanger, Gasser and Erlanger were awarded a Nobel prize in 1944.
As an administrator, Gasser continued Flexner's practice of allowing researchers to work in freedom.
“About new knowledge two points are clear”, he wrote. “It cannot be forecast; and it cannot be achieved through administrative action. All that can be done is to create optimal conditions for its production.”
(The Rockefeller university achievements 1901-2001: a century of science for the benefit of humankind, by Elizabeth Hanson, introduction by Arnold J. Levine, epilogue by David Rockefeller, 2000, )
____________________________________
Creativity is people-based
The spark of creativity burns fleetingly in people in strange and unpredictable ways. Creativity cannot be forced, it can only be encouraged. Setting an atmosphere and environment that encourages the staff to think broadly, and to reach for new thoughts is a principal task of lab management. From Alice in Wonderland: "Think 400 impossible thoughts before breakfast." Creativity is bestowed on people in different proportions.
Implied teaching role
As a developer or importer of new behavior-changing ideas for a company, a
research lab is inevitably in the teaching business. Teaching and research have had a long and productive association in the academic world, and it is equally true in the contemporary industrial world. Part of my recruiting description for Sun Labs has been that I want new staff to come to the lab where their job will be to develop interesting curricula to teach the fruits of their creativity to the company.
Of course, they must create something new and valuable through their research activities as the topical basis for such curricula, but their success is not solely determined by their innate creativity. The best idea in the world contained only in a small group of lab heads is worth little to the company. The job of the lab is to infect the company with new knowledge—often in spite of people's resistance to change and learning.
source:
[41] W. R. (Bert) Sutherland, “Management of industrial research : exploring the unknown technical future”, Perspective series 2008─7, Sun labs, July 2008.
(Lynn Conway, Reminiscences of the VLSI revolution : how a series of failures triggered a paradigm shift in digital design, Fall 2012, IEEE solid-state circuits magazine. [2012])
References
[6] H. Garfinkel, Studies in ethnomethodology, Prentice hall, Englewood cliffs, N.J., 1967.
[19] I. Sutherland, C. Mead and T. E. Everhart, “Basic limitations in microcircuit fabrication technology”, ARPA report R-1956-ARPA, published by Rand corporation, Santa Monica, CA, November 1976.
[20] I. Sutherland, “The problem : how to build digital electronic circuits from now to 1985”, Letter to W. R. Sutherland describing the challenges presented by advances in microelectronics and proposing the Xerox-Parc/Caltech collaboration, January 26, 1976.
[21] L. Conway, Ed., “The VLSI archive : an online archive of documents and artifacts from the Mead-Conway VLSI design revolution”, lynnconway.com. http://ai.eecs.umich.edu/people/conway/VLSIarchive.html
[31] mosis.com
[32] M. Marshall, L. Waller, and H. Wolff,
For optimal VLSI design efforts, Mead and Conway have fused devise fabrication and system-level architecture.
[33] E. M. Roger, Diffusion of innovations, Free press, New York, 1962.
[34] Computer science and telecommunications board, National research council, Funding a revolution: government support for computing research, National academy press, Washington, DC, 1999, pp.113─122.
[35] M. Stefik and L. Conway, “Towards the principled engineering of knowledge”, AI magazine, vol. 3:3, pp.4─16, summer 1982.
[41] W. R. (Bert) Sutherland, “Management of industrial research : exploring the unknown technical future”, Perspective series 2008─7, Sun labs, July 2008.
[42] W. R. (Bert) Sutherland, “Faith, Funds, & Fate: prerequisites for the development and transfer of new technology”, Presentation at the celebration of the 40th anniversary of USC-ISI, Marina del Rey, California, April 26, 2012.
[43] L. Conway, “The VLSI archive”, Electronic design news, June 3, 2009. http://edagraffiti.com/?p=101
(Lynn Conway, Reminiscences of the VLSI revolution : how a series of failures triggered a paradigm shift in digital design, Fall 2012, IEEE solid-state circuits magazine. [2012])
____________________________________
Stoic at work : ancient wisdom to make your job a bit less annoying
by Annie Lawson
with illustrations by Oslo Davis
2023
text copyright by Annie Lawson 2023
p.140
Being able to switch between focus and daydreaming is an important skill, but it is curtailed by constant busyness. Engineering scarcity into our days can free up time for creative thought. [[ Engineering scarcity ? ]] Many important discoveries were made during downtime. Nikola Tesla had an insight into rotating magnetic fields while walking in Budapest. Albert Einstein liked to listen to Mozart during his breaks from intense thinking sessions.
Salvador Dalí believed he got a creativity boost during the early stage of sleep, when reality gets mixed up with fantasy. To use the technique, Dalí would hold an object, such as a spoon or a ball, while falling asleep in a chair. As he drifted off, the object would fall, make a noise and wake him up. Having spent a few moments on the brink of unconsciousness, he would be ready to start work.
Stoic at work : ancient wisdom to make your job a bit less annoying
by Annie Lawson
with illustrations by Oslo Davis
____________________________________
Oral History of Barry Boehm, part 2 of 2
Recorded February 20, 2018
[ software development ]
https://archive.computerhistory.org/resources/access/text/2022/07/102738733-05-01-acc.pdf
([ software is ... like ... the glue ])
Winston Royce, 1970, wrote a definitive paper for an electronics conference
described the waterfall model,
you really want to do some building it twice, so that you know roughly the directions you want to go.
prototyping is a form of risk reduction,
you want to be doing a combination of specification and prototypes.
process workshops
you were not going step-by-step, but you were going around a bunch of spiral circles.
the spiral model
got some better characteristics than the waterfall.
more risk reduction
good for big defense systems but heavyweight for business systems and the like.
agile method, an intial prototype.
“What you want to do is just interact with your customer to find out what they want and build them an initial prototype. What they don't like about the prototype, you do another sprint and fix that and say now what do you want.”
V-model, the waterfall tipped over.
Watts Humphrey
He was not particularly interested in how you build software, just whether your projections about cost and budget, scheduling and budget, could be trusted or not. His claim was, his motivation for the CMM, was just so that
the Defense Department knew which of those lying contractors could be believed, and which ones could not be believed.
If you got to Level 3 then you probably could make believable predictions. Then at that point, you're okay with Watts.
TRW
creating a new satellite system
the propulsion people
the structures people
the guidance and control people
the communication people
architectures
the software people
there are combinations of agile things that you want to do, and plan-driven things that you want to do.
Another book that I ended up co-authoring with a guy named
Rich [Richard] Turner was something called
Balancing Agility and Discipline.
It basically said that there are things where lockstep discipline is not a very good thing to do, but there are places where coordinating what you're doing is a good thing to do. Showing some examples of where the “Agile Manifesto”—they had a manifesto, that basically said things like, “Embracing change rather than following a plan.”
At one point at TRW, I was on a panel that was saying, “What were the causes of so many missiles getting launched from Vandenberg and then blowing up because of the software?”
In most cases, it was because people are responding to change over following a plan and saying, "We've got this fix that, we've got to do, or we've got this telemetry station that's moved and we've got to put a patch in the software. There's not enough time to do the regression testing and the configuration management and following the plan."
regression testing and the configuration management
And so, launched the rocket and boom, there it goes. Responding to change over following a plan may not be good in some situations.
sometimes the rocket blows up and sometimes it doesn't, and I want to be more predictable than that.
in fact for things that really matter, if your whole bank is going to rest on this or you what they have, and they want to be sure they can trust it.
Hubert Dreyfus, who wrote the book What Computers Can't Do, and showed all of the failed predictions that say, “In 1958, in 10 years, the computer will be the world's chess champion.” Well, they got there but not in ten years. I was getting sort of a balance of skepticism and enthusiasm.
Computer Systems Analysis Organization at RAND.
to apply system analyses to various challenges ── One of them was the Minuteman command and control system.
missiles that were in bunches in Montana, North Dakota, and various places.
People were worried about the fact that if the Russians targeted their missiles the right way they could disrupt the communication system so that the Minuteman wouldn't be able to launch.
finding that you needed a learning algorithm to determine if you had a system where parts of the system were going down, you needed some way to reroute things that weren't the shortest path until one of the thin nodes on the shortest path disappeared.
long, long ago for the first time, my boss said "Your job as a manager is to manage expectations. Never let people's expectations get out of the box, because if they get out of the box you can never win. If you do those incredible things people will say, ‘Well sure,’ but most likely you're not going to be able to do those things and people are going to get mad."
in 1955, “Pretty soon a computer's going to be the world's best chess player. Computers are going to automatically translate any language into any other language faster than anybody can even think the words.” Said all this crazy stuff, and it's just inevitable that at some point these promises are not going to be met and then there's a backlash and there's an AI winter.
1955
____________________________________
expense reducing technology
A dollar spent in the research lab for new product may be returned to the
company only after many years and much additional investment expanse produce
something that customers will buy in volume. The dollar spent in the research lab for some internal expense-reducing technology can be returned to the bottom line in the first year that the improvement is adopted in the company. The lab portfolio of activities must balance these two aspects of technology utilization.
Avoiding technical tarpits
In a similar vein, if the lab can clearly demonstrate what technology is not yet ready for prime time, its sponsor can avoid difficulty, expense and even
embarrassment. Several years ago, the digital transmission technology of
Asynchronous Transfer Mode (ATM) was the rage and widely touted for rapid
adoption to the desktop. A Sun Labs project tried to use ATM technology for our own building's internal network and found we could not make it work well
enough. This seeming failure helped change an incipient Sun campus network
plan from ATM to IP technology and avoid an expensive mistake.
a viable promising project ready to go with a champion is more concrete than ...
source:
[41] W. R. (Bert) Sutherland, “Management of industrial research : exploring the unknown technical future”, Perspective series 2008─7, Sun labs, July 2008.
(Lynn Conway, Reminiscences of the VLSI revolution : how a series of failures triggered a paradigm shift in digital design, Fall 2012, IEEE solid-state circuits magazine. [2012])
____________________________________
August 23, 2013
The Berkeley Par Lab:
Progress in the Parallel Computing Landscape
David Patterson, Dennis Gannon, and Michael Wrinn
Editors
• The Network of Workstations (NOW) project, 1993-1998, developed NOW-I and NOW-II, which were clusters of workstations that proved valuable in applications ranging from encryption to sorting. The Inktomi search engine was first built on NOW II, which led to a search engine startup company. Inktomi Inc. in turn demonstrated to the fledgling Internet industry the value of clusters of a large number of low-cost computers versus fewer more expensive high-end servers, which Google and others followed.
Project alumni became faculty at Berkeley, Harvard, Illinois, Princeton, Rutgers, Stanford, Texas, and Wisconsin. Tom Anderson, Eric Brewer, David Culler, and I led the NOW project. 7
• NOW was controversial in that it argued for building scalable large-scale computers from standard networks and computers as compared to distributed, cache-coherent, non-uniform memory access machines (CC-NUMA) as advocated by the Stanford DASH project and later embodied in the SGI Origin computers.
[7] A. Fox and D. Patterson. Engineering software as a service: An agile approach using cloud computing. Strawberry Canyon, 2013.
____________________________________
re-invention (Rogers, Everett M., Diffusion of innovation., 1. diffusion of innovation., 2. diffusion of innovations──study and teaching──history.)
Rogers, Everett M.
Diffusion of innovation.
rev. ed. of : Communication of innovations. 2nd ed. 1971.
bibliography
includes indexes
1. diffusion of innovation.
2. diffusion of innovations──study and teaching──history.
HM101.R57 1983
303.4'84
____________________________________
Mix it up ("high-low mix" concept)
The "high-low mix" concept, born out of the group of Air Force reformers sometimes called the "Fighter Mafia"—Cols. John Boyd, Thomas Christie, Franklin “Chuck” Spinney, and Pierre Sprey—sought to fix the death spiral of fighter aircraft development cost and complexity by augmenting a few expensive, high-capability aircraft built for air superiority with a larger number of less-complicated and less-expensive aircraft that could handle basic air defense, strike missions, and close air support.
In the 1970s, the "high" was the F-15 Eagle, and the "low" was the multirole F-16 Fighting Falcon. The Navy followed suit with the F-14 Tomcat as its "high" and the F/A-18 Hornet as a less-expensive "low."
To fix the death spiral of fighter aircraft development cost
• a few expensive, high-capability aircraft built for air superiority
• "high" was the F-15 Eagle
• a larger number of less-complicated and less-expensive aircraft that could handle basic air defense, strike missions, and close air support.
• "low" was the multirole F-16 Fighting Falcon
source:
https://www.textise.net/showText.aspx?strURL=https://arstechnica.com/tech-policy/2021/04/sitrep-is-the-f-35-now-officially-a-failure/
____________________________________
Steve Cross
One of my program managers was a Air Force major when I got there, and he got promoted to lieutenant colonel, but he introduced himself as saying, "I am the major cross that you're going to have to bear.”
He got some CMU [Carnegie Mellon University] and MIT people to come up with an AI constraint-based planning approach to solve transportation problems. This was in 1990 and '91, and they came up with a system that could do in four hours using constraint-based planning what it was taking the clunky transportation command software four days to do. Just about in 1991, we needed to get a half a million people off to the Middle East to fight the first Iraq war. And the transportation command said, “We're confiscating your Sun computers because we need your system to plan all of these things.”
Steve Cross got the Golden Nugget Award from the commander of the Air Force and went on from there.
Unfortunately, what I found was that a lot of organizations didn't want money added to their budget that they didn't control.
the two-star admiral who was the head of the Office of Naval Research.
"I get my money from the Chief of Naval Operations and I follow his priorities, and our big priority right now is corrosion. Our boats are getting corroded and we need more research in corrosion technology. And software, I can't really accept your software money. If I get more money I'm going to use it for corrosion."
DevOps [development operations]
in getting companies—the Defense Department to build things more like what the really advanced software companies like Amazon are doing. Amazon comes out with a new release of their software every 11 seconds, and there's this approach called DevOps [development operations] that fundamentally says that you organize your system in a way that many things can be going on in parallel, many things can be integrated in parallel, many things can be prioritized in terms of rushing testing and the like. So that the most important things that need to be in there, the next 11 seconds are going to be there, and then the flexible plan for what's going to happen in the next 11 seconds, and then the next hour,
Dean Leffingwell, who of built something called SAFe [Scaled Agile Framework], for “structured agile” or something.
Chalmers University in Sweden, Jan Bosch, who has made a success of doing this kind of thing, and has got a consortium of big European manufacturing companies, like BMW and Volkswagen and Saab and Volvo and Ericsson and the like.
Simon Ramo, who was the “R” in “TRW,” to help the company recruit more really bright software people because that was really a limiting feature on many of
the things that the company was trying to do.
to come up with a program where TRW would hire some really bright people, and give them two days a week to go to either USC or UCLA to get a master's degree. This would be a plum thing that really smart people would do.
UCLA and USC
For one, they were having a hard time hiring U.S. citizens.
Simon Ramo said, “We don't want to get super programmers out of this. What I found in our company the most valuable people we have are T-shape people. They're very deep in whatever their specialty is.
Maybe it's astrodynamics and maybe it's electronics, maybe it's communications, but maybe it's software. But the ones that are really helpful are the ones that know a little bit more about the other parts of it, about the management and the economics and all the other things.”
This was the basically true of most of the new hires that we would get as software engineers at TRW. It would take a year to get them to appreciate and understand what configuration management was all about, what requirements engineering was all about, how do you estimate costs of things and the like.
when I was 55 and saying, "I'm going to early retire or stay here for another ten years and get excused at 65, I think what I'd like to do is go to either UCLA or USC and take these I-shape bachelor's students in computer science and make T-shaped people out of them.” The software projects were a good example of that because they had to learn the requirements are things you negotiate. They aren't things that you just write up and follow.
basically they knew what configuration management was all about.
I think the main thing is that change is going to be constant.
“We want you to not just learn, we want you to learn how to learn.”
Software Management and Economics course
“You are the CTO [chief technical officer] of a 500 person software company and your chief executive officer is a concerned about AI, or DevOps, or Artificial Intelligence of various kinds and the like. What you need to do is to give him an analysis of how mature are these, and what are their strengths and what are their weaknesses, and what would we have to do to address these. And that you're going to get graded on how incisive your analysis is, plus the number of different ways that you learn about things. So you can't just go to Google and stop there. You should try to interview some people that are in companies, or are developing this kind of research and things like that. You should look at the proceedings of conferences and the ACM [Association for Computing Machinery]/IEEE [Institute of Electrical and Electronics Engineers] literature, and so the more sources that you do the better your grade is going to be.”
We've found that we have to learn
how to learn in running the course.
International Conference on Software Engineering (ICSE)
source:
Oral History of Barry Boehm, part 2 of 2
https://archive.computerhistory.org/resources/access/text/2022/07/102738733-05-01-acc.pdf
interviewd by :
David C. Brock
Lee Osterweil
Recorded February 20, 2018
____________________________________
Jon Gernter., The idea factory : the Bell Labs and the great age of American innovation, [2012]
p.51
In truth, large leaps forward in technology rarely have a precise point of origin. At the start, forces that precede an invention merely begin to align, often imperceptibly, as a group of people and ideas converge, until over the course of months or years (or decades) they gain clarity and momentum and the help of additional ideas and actors. Luck seems to matter, and so does timing, for it tends to be the case that the right answers, the right people, the right place ── perhaps all three ── require a serendipitous encounter with the right problem. And then ── sometimes ── a leap. Only in retrospect do such leaps look obvious. When Niels Bohr ── along with Einstein, the world's greatest physicist ── heard in 1938 that splitting a uranium atom could yield a tremendous burst of energy, he splapped his head and said, “Oh, what idiots we have been!”11
p.107
John Bardeen, the most careful of men, referred to his transitor work as a “discovery” of “transistor action”; he and Brattain had effectively observed in their experiment how a current applied to a slightly impure slice of germanium could hasten the movement of microscopic holes inside and thus amplify a signal.
p.107
a product like the transistor could ultimately fail for technical reasons (if it proved unreliable) or for manufacturing reasons (if it proved difficult to reproduce consistently or cheaply). Also, it might be the case that there was no market for a new device:
p.108
“But learning how to make them by the hundreds or thousands, and of sufficient uniformity to be interchangeable and reliable, was another problem.”31
p.108
Jack Morton
innovation was “a total process” of interrelated parts.
pp.108─109
“It is not just the discovery of new phenomena, nor the development of a new product or manufacturing technique, nor the creation of a new market. Rather, the process is all these things acting together in an integrated way toward a common industrial goal.”33
p.109
“reliability”, “reproducibility”, “designability”
pp.128─129
Rather, one might think of [information] in term of its ability to resolve uncertainty. Information provided a recipient with something that was not previously known, was not predict able, was not redundant. “We take the essence of information as the irreducible, fundamental underlying uncertainty that is removed by its receipt,” a Bell Labs executive named Bob Lucky explained some years later.30
(The idea factory : the Bell Labs and the great age of American innovation / Jon Gernter.
1. bell telephone laboratories──history──20th century.
2. telecommunication──united states──history──20th century.
3. technological innovations──united states──history──20th century.
4. creative ability──united states──history──20th century.
5. inventors──united states──history──20th century.
TK5102.3.U6G47 2012
384──dc23
384 Gernter
)
____________________________________
·‘’•─“”
<------------------------------------------------------------------------>
πόλλ' οἶδ' ἀλώπηξ,ἀλλ' ἐχῖνος ἓν μέγα πόλλ' οἶδ' ἀλώπηξ,ἀλλ' ἐχῖνος ἓν μέγα
____________________________________
*2 “This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold with the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional person should be sought.”
──From a Declaration of Principles jointly adopted by a Committee of the American Bar Association and a Committee of Publishers and Associations
(Ackoff's best : his classic writings on management, Russell L. Ackoff., © 1999, hardcover, John Wiley & Sons, Inc., p.139)
“This [copy & paste reference note] is designed to provide accurate and authoritative information in regard to the subject matter covered. It is [archive] with the understanding that the [researcher, investigator] is not engaged in rendering professional services. If professional advice or other
expert assistance is required, the services of a competent professional person should be sought.”
──From a Declaration of Principles jointly adopted by a Committee of the American Bar Association and a Committee of Publishers and Associations
--
No part of this publication may be reproduced, stored in, or introduced into a retrieval system, or transmitted, in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), without the prior permission of the publisher.
The W. Edwards Deming Institute. All rights reserved. Except as permitted under the United States copyright act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher.
All right reserved. No part of this book may be reproduced without written permission from the publisher, except by a reviewer who may quote brief passages or reproduce illustrations in a review with appropriate credits nor may any part of this book be reproduced, stored in a retrieval system, or transmitted in any form or by any means ── electronic, mechanical, photocopying, recording, or other ── without written permission from the publisher.
The information in this book is true and complete to the best of our knowlege. All recommendations are made without guarantee on the part of the author or Storey publishing. The author and publisher disclaim any liability in connection with the use of this information.
NOTICE: In accordance with Title 17 U.S.C., section 107, some material is provided without permission from the copyright owner, only for purposes of criticism, comment, scholarship and research under the "fair use" provisions of federal copyright laws. These materials may not be distributed further, except for "fair use," without permission of the copyright owner. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml
notice: Do not purchase this book with the hopes of curing cancer or any other chronic disease
We offer it for informative purposes to help cope with health situations and do not claim this book furnishes information as to an effective treatment or cure of the disease discussed ─ according to currently accepted medical opinion.
Although it is your right to adopt your own dietary and treating pattern, never the less suggestions offered in this book should not be applied to a specific individual except by his or her doctor who would be familiar with individual requirements and any possible complication. Never attempt a lengthy fast without competent professional supervision.
the home health handbook makes every effort to insure that its information is medically accurate and up-to-date. However, the information contained in this handbook is intended to complement, not substitute for, the advice of your own physician. Before embarking on any medical treatment or changing your present program, you should consult with your doctor, who can discuss your individual needs, symptoms and treatment.
All right reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without prior permission in writing from the publisher. The Australian copyright act 1968 (the Act) allows a maximum of one chapter or 10 per cent of this book, which ever is the greater, to be photocopied by any educational institution for its educational purposes provided that the educational institution (or body that administers it) has given a renumeration notice to the copyright agency (Australia) under the Act.
We acknowledge that we meet and work on the traditional lands of the Cammeraygal people of the Eora nation and pay our respects to their elders past, present and future.
── And that was true even with a team that included such defense giants as SAIC, Boeing, CSC, AT&T, and Booz Allen Hamilton. We were also trying to do too much, too quickly.
── We would have been better advised to pick our spots and work incrementally, trusting to spiral development to eventually get us to where we wanted to be.
── (Michael V. Hayden, Playing to the edge : American intelligence in the age of terror, 2016, pp.20-21)
Michael V. Hayden, Playing to the edge : American intelligence in the age of terror, 2016
p.20
We found that when we went to industry for things they already knew how to do, we got impressive results. When we went to them for things nobody had done yet, we found that at best they weren't much better or faster than we were. And that was true even with a team that included such defense giants as SAIC, Boeing, CSC, AT&T, and Booz Allen Hamilton.
We were also trying to do too much, too quickly. Trailblazer comprised multiple moon shots.
([ DARPA - learning curve ])
p.21
We would have been better advised to pick our spots and work incrementally, trusting to spiral development to eventually get us to where we wanted to be.
(Playing to the edge : American intelligence in the age of terror / Michael V. Hayden, New York : Penguin Press, 2016, (hardback) (ebook), intelligence service──united states. | national security──united states. | united states. central intelligence agency. | united states. national security agency. | biography & autobiography / political. | political science / political freedom & security / intelligence. | history / united states / 21st century., JK468.I6 H39 2016 (print), JK468.I6 (ebook), 327.1273──dc23, 2016, )
[[ manufacturing ]]
[[ prototype ]]
[[ how many prototype did you build before you are satisfy with product ]]
____________________________________
Ashton Carter, Inside the five-sided box : lessons from a lifetime of leadership in the Pentagon, [2019]
p.324
The biggest challenge was connecting DARPA's breakthroughs to real military hardware. The “valley of death” that separates R&D from production is an age-old problem in technology, and bridging it is a vital challenge that every senior tech manager must tackle.
p.325
that divides research and engineering from acquisition in two separate organizations!
You might as well write a statute repealing the laws of thermodynamics (which explain why a perpetual motion machine is impossible).
( Inside the five-sided box: lessons from a lifetime of leadership in the Pentagon / Ash Carter.; Ashton B. Carter, author.; [2019]; subjects: Carter, Ashton B. | united states. department of defense. | united states ──military policy ── decision making. | pentagon (va.) | united stats. department of defense ── history. | united states. department of defense ── procurement. | united states. department of defense ── officials and employees ── biography. | cabinet officers ── united states ── biography.; UA23.C2746 2019; DDC 355.6092 [B]──dc23 )
____________________________________
Jon Gernter., The idea factory : the Bell Labs and the great age of American innovation, [2012]
p.22
Then Jewett asked his friend for help. “Let us have one or two, or even three, of the best of the young men who are taking their doctorates with you and are intimately familiar with your field. Let us take them into our laboratory in New York and assign to them the sole task of developing a telephone repeater.”18
Here was a new approach to solving an industrial problem, an approach that looked not to engineers but to scientists.
p.23
Within two years Arnold came up with several possible solution to the repeater problem, but he mainly went to work on improving an amplifier known as the audion that had been brought to AT&T in 1912 by an independent, Yale-trained inventor named Lee De Forest. The early audion was vaguely magical. It resembled a small incandescent light bulb, yet instead of a hot wire filament strung between two supporting wires it had three elements ── a metal filament that would get hot and emit electrons (called a cathode); a metal plate that would stay cool and attract electrons (called an anode); and between them a wire mesh, or “grid”. A small electrical current, or signal, that was applied to the audion's grid could be greatly amplified by another electrical current that was traveling from the hot cathode to the cool anode. Arnold found, through trial and error, the best materials, as well as a superior way to evacuate the air inside the audion tube. (He suspected correctly that a high vacuum would greatly improve the audion's efficiency.) Once Arnold had refined the audion, he, Jewett, and Millikan convened in Philadelphia to test it against other potential repeater ideas. The men listened in on phone conversations that were passed through the various repeaters, and they found the audion clearly superior. Soon to be known as the vacuum tube, it and its descendants would revolutionize 20th century communications.
p.32
That an industrial laboratory would focus on research and development was not entirely novel; a few large German chemical and pharmaceutical companies had tried it successfully a half century before. But Bell Labs seemed to have embraced the idea on an entirely different scale.
p.32
Harold Arnold
As Arnold explained, his department would include, “the fields of physical and organic chemistry, of metallurgy, of magnetism, of electrical conduction, of radiation, of electronics, of acoustics, of phonetics, of optics, of mathematics, of mechanics, and even of physiology, of psychology, and of meteorology.”10
p.51
In truth, large leaps forward in technology rarely have a precise point of origin. At the start, forces that precede an invention merely begin to align, often imperceptibly, as a group of people and ideas converge, until over the course of months or years (or decades) they gain clarity and momentum and the help of additional ideas and actors. Luck seems to matter, and so does timing, for it tends to be the case that the right answers, the right people, the right place ── perhaps all three ── require a serendipitous encounter with the right problem. And then ── sometimes ── a leap. Only in retrospect do such leaps look obvious. When Niels Bohr ── along with Einstein, the world's greatest physicist ── heard in 1938 that splitting a uranium atom could yield a tremendous burst of energy, he splapped his head and said, “Oh, what idiots we have been!”11
p.113
There had been whispers in the electronics industry about whether Bell Labs' enthusiasm over the transistor was overblown; the reported difficulty in manufacturing the devices only added to the skepticism. Whether it was a shortcoming or an advantage, Kelly's confidence was almost certainly rooted in his early experiences. He remembered the endless days and nights constructing vacuum tubes in lower Manhattan, the countless problems in the beginning and then the stream of incremental developments that improved the tubes' performance and durability to once-unimaginable levels. He could remember, too, that as the tubes became increasingly common ── in the phone system, radios, televisions, automobiles, and the like ── they had come down to price levels that once seemed impossible. He had long understood that innovation was a matter of economic imperatives. As Jack Morton had said, if you hadn't sold anything you hadn't innovated, and without an affordable price you could never sell anything. So Kelly looked at the transistor and saw the past, and the past was tubes. He thereby intuited the future.
pp.149─150
Mervin Kelly
March 23, 1950
a polished version of the lecture about Bell Labs
Royal society
Bell Labs was the world's foremost example of a place where scientists pursued creative technology.
p.150
Bell Labs helped maintain and improve that system, he said, by creating an organization that could be divided into three groups. The first group was research, where scientists and engineers provided “the reservior of completely new knowledge, principles, materials, methods and art.” The second group was in systems engineering, a discipline started by the Labs, where engineers kept one eye on the reservoir of new knowledge and another on the existing phone system and analyzed how to integrate the two. In other words, the systems engineers considered whether new applications were possible, plausible, necessary, and economical. That's when the third group came in. These were the engineers who developed and designed new devices, switches, and transmissions systems. In Kelly's sketch, ideas usually moved from (1) discovery, to (2) development, to (3) manufacture.
p.150
the telephone system, Mervin Kelly
p.151
a living organism
In truth, the handoff between the three departments at Bell Labs was often (and intentionally) quite casual. Part of what seemed to make the Labs “a living organism,” Kelly explained, were social and professional exchanges that moved back and forth, in all directions, between the pure researchers on one side and the applied engineers on the other. These were formal talks and informal chats, and they were always encouraged, both as a matter of policy and by the inventive design of the Murray Hill building. Researchers and engineers would find themselves discussing their respective problems in the halls, over lunch, or they might be paired together on a project, either at their own request or by managers. Or a staffer with a question would casually seek out an expert, “whether he be a mathematician, a metallurgist, an organic chemist, an electromagnetic propagation physicist, or an electron device specialist.”
p.151
Physical proximity, in Kelly's view, was everything. People had to be near one another. Phone calls alone wouldn't do.
Kelly had even gone so far as to create “branch laboratories” as Western Electric factories so that Bell Labs scientists could get more closely involved in the transition of their work from development to manufacture.
p.152
What he went on to describe in London, though, was a systematized approach to innovation, the fruit of three decades of consideration at the Labs. To Kelly, inventing the future wasn't just a matter of inventing things for the future; it also entailed inventing ways to invent those things.
Bell Labs' experience over the past few years demonstrated that the process of innovation could now be professionally fostered and managed with a large degree of success ── and even, perhaps, with predict ability.
Industrial science was now working on a scale, and embracing a complexity,
p.152
He had a formula
p.152
In TECHNOLOGY, the odds of making something truly new and popular have always tilted toward failure.
(The idea factory : the Bell Labs and the great age of American innovation / Jon Gernter.
1. bell telephone laboratories──history──20th century.
2. telecommunication──united states──history──20th century.
3. technological innovations──united states──history──20th century.
4. creative ability──united states──history──20th century.
5. inventors──united states──history──20th century.
TK5102.3.U6G47 2012
384──dc23
384 Gernter
)
____________________________________
National academy of sciences
A biographical memoir
of
Mervin Joe Kelly (1894─1971)
by John R. Pierce
copyright 1975
Mervin Joe Kelly (February 14, 1894─March 18, 1971)
creative technical management.
As Frederick R. Kappel, former board chairman of AT&T said after
Kelly's death:
"He was a great fellow for the Bell System. Mervin was
always and forever pushing the operating management, and the
heads of AT&T as well, to get on with new things. His aggres-
siveness got him in a lot of hot arguments, but I always sat back
and said, 'Give it to them, Mervin, that's what we need.' Every
place needs a fireball or spark plug, and he was it."
Others were less disturbed by Kelly's temper. Estill Green
describes his experience as vice president in charge of systems
engineering in these mellow words:
"A few years in close association with Mervin were the hap-
piest time of my life. For years on end I had believed I needed
insulation from the high voltage. Yet when I was directly ex-
posed to it, I never experienced a serious shock, and I rejoiced
to observe how the high potential overpowered inertia and loose
thinking and prejudice.
"I learned never to oppose him when he had the bit in his
teeth. Next morning I could remark casually, 'Mervin, there
are some aspects of that matter discussed in yesterday's confer-
ence that you may not be fully aware of.' He would listen, and
generally modify his position, to a minor or sometimes major
extent."
In 1943 Kelly outlined a branch-laboratory concept. This
eventually led to the establishment of laboratories for final
development at manufacturing locations of Western Electric.
This proved important in several ways. It linked final develop-
ment and its procedures and personnel closely to those respon-
sible for the manufacture of new devices and systems. It pre-
vented too large a concentration of personnel in a few central
locations. It gave a desirable measure of responsibility and
independence to work in various well-defined fields of devel-
opment.
'organized creative technology'
While Kelly recognized basic research as the source of all
technological advances, he understood that a complicated tech-
nological process lies between discovery and use. He wrote:
"There has been so much emphasis on industrial research
and mass-production methods in my country, that even our
well-informed public is not sufficiently aware of the necessary
and most important chain of events that lies between the initial
step of basic research and the terminal operation of manufac-
ture. In order to stress the continuity of procedures from re-
search to engineering of product into manufacture and to
emphasize their real unity, I speak of them as the single entity
'organized creative technology'."
Using the Bell Laboratories as an example of organized tech-
nology, Kelly delineated three areas that preceded the manu-
facture of complicated technological systems:
"The first includes all of the research and fundamental
development. This is our non-scheduled area of work. It pro-
vides the reservoir of completely new knowledge, principles,
materials, methods, and art that are essential for the develop-
ment of new communications systems and facilities.
"The second we call 'systems engineering'. Its major respon-
sibility is the determination of the new specific systems and
facilities development projects—their operational and economic
objectives and the broad technical plan to be followed. 'Systems
engineering' controls and guides the use of the new knowledge
obtained from the research and fundamental development pro-
grams in the creation of new telephone services and the im-
provement and lowering of cost of services already established.
"The third encompasses all specific development and design
of new systems and facilities. The work is most carefully pro-
grammed in conformity with the plan established by the systems
engineering studies. Our research and fundamental develop-
ment programs supply the new knowledge required in meeting
the objectives of the new specific devleopments."
Concerning Systems Engineering, Kelly said:
"Approximately 10% of our scientific and technical staff are
allotted to systems engineering. Its staff members must supply
a proper blending of competence and background in each of
the three areas that it contacts: research and fundamental devel-
opment, specific systems and facilities development, and opera-
tions. It is therefore, largely made up of men drawn from these
areas who have exhibited unusual talents in analysis and the
objectivity so essential to their appraisal responsibility."
Kelly illustrated an ideal relation between systems engi-
neering, research and development by the case of the NIKE
antiaircraft missile:
"For example, the programming study on the NIKE missile
system established that basic knowledge and art were available
for the development of a system that would meet the service
requirements except for a particular area of radar technology.
This area was at once subjected to a research and exploratory
development attack. The project was not undertaken until this
deficiency was eliminated by new knowledge from research.
The NIKE missile system now in production meets the require-
ments initially agreed upon and in its technical character is in
close correspondence with the plan of the initial study.
"I am familiar with large military systems developments
where this approach is absent, where research and exploration
are intermingled with specific development, probably with the
intent of gaining time. Actually, time has been lost."
Kelly goes on to say that development, while a continuous
operation, is done in three distinct stages: first the laboratory
model; after tests and modifications, the preproduction model,
which is field tested; and finally, the final design for manufac-
ture (by Western Electric).
Above all, a technological organization must have the lead-
ership to see and pursue real opportunities and real needs. In
an address to a naval research conference, Kelly said:
"The first, and perhaps the most important, factor is the
program itself. What shall it contain? What can be discarded
at once, and what shall be eliminated after limited exploration?
How can comprehensive coverage with freedom from gaps be
assured? In an endeavor so broad in scope and requiring such
a highly functional organization for its operation, how can
unneeded duplication be prevented, and duplication that is
worthwhile, though usually small in volume, be provided?"
Leaders or
managers must be technologically trained and technologically
competent. Only thus can decisions be based on insight and
understanding rather than on salesmanship and hearsay.
source:
National academy of sciences
A biographical memoir
of
Mervin Joe Kelly (1894─1971)
by John R. Pierce
copyright 1975
Mervin Joe Kelly (February 14, 1894─March 18, 1971)
____________________________________
[ new knowledge ]
“About new knowledge two points are clear. It cannot be forecast; and it cannot be achieved through administrative action. All that can be done is to create optimal conditions for its production.”
The Rockefeller university achievements 1901-2001
The Rockefeller university achievements
a century of science for the benefit of humankind
1901-2001
by Elizabeth Hanson
introduction by Arnold J. Levine
epilogue by David Rockefeller
2000
p.75
Herbert Spencer Gasser
electrophysiological techniques for studying nerves.
studies into the nature of nerve conduction, carried out with Joseph Erlanger, Gasser and Erlanger were awarded a Nobel prize in 1944.
As an administrator, Gasser continued Flexner's practice of allowing researchers to work in freedom.
“About new knowledge two points are clear”, he wrote. “It cannot be forecast; and it cannot be achieved through administrative action. All that can be done is to create optimal conditions for its production.”
(The Rockefeller university achievements 1901-2001: a century of science for the benefit of humankind, by Elizabeth Hanson, introduction by Arnold J. Levine, epilogue by David Rockefeller, 2000, )
____________________________________
Creativity is people-based
The spark of creativity burns fleetingly in people in strange and unpredictable ways. Creativity cannot be forced, it can only be encouraged. Setting an atmosphere and environment that encourages the staff to think broadly, and to reach for new thoughts is a principal task of lab management. From Alice in Wonderland: "Think 400 impossible thoughts before breakfast." Creativity is bestowed on people in different proportions.
Implied teaching role
As a developer or importer of new behavior-changing ideas for a company, a
research lab is inevitably in the teaching business. Teaching and research have had a long and productive association in the academic world, and it is equally true in the contemporary industrial world. Part of my recruiting description for Sun Labs has been that I want new staff to come to the lab where their job will be to develop interesting curricula to teach the fruits of their creativity to the company.
Of course, they must create something new and valuable through their research activities as the topical basis for such curricula, but their success is not solely determined by their innate creativity. The best idea in the world contained only in a small group of lab heads is worth little to the company. The job of the lab is to infect the company with new knowledge—often in spite of people's resistance to change and learning.
source:
[41] W. R. (Bert) Sutherland, “Management of industrial research : exploring the unknown technical future”, Perspective series 2008─7, Sun labs, July 2008.
(Lynn Conway, Reminiscences of the VLSI revolution : how a series of failures triggered a paradigm shift in digital design, Fall 2012, IEEE solid-state circuits magazine. [2012])
References
[6] H. Garfinkel, Studies in ethnomethodology, Prentice hall, Englewood cliffs, N.J., 1967.
[19] I. Sutherland, C. Mead and T. E. Everhart, “Basic limitations in microcircuit fabrication technology”, ARPA report R-1956-ARPA, published by Rand corporation, Santa Monica, CA, November 1976.
[20] I. Sutherland, “The problem : how to build digital electronic circuits from now to 1985”, Letter to W. R. Sutherland describing the challenges presented by advances in microelectronics and proposing the Xerox-Parc/Caltech collaboration, January 26, 1976.
[21] L. Conway, Ed., “The VLSI archive : an online archive of documents and artifacts from the Mead-Conway VLSI design revolution”, lynnconway.com. http://ai.eecs.umich.edu/people/conway/VLSIarchive.html
[31] mosis.com
[32] M. Marshall, L. Waller, and H. Wolff,
For optimal VLSI design efforts, Mead and Conway have fused devise fabrication and system-level architecture.
[33] E. M. Roger, Diffusion of innovations, Free press, New York, 1962.
[34] Computer science and telecommunications board, National research council, Funding a revolution: government support for computing research, National academy press, Washington, DC, 1999, pp.113─122.
[35] M. Stefik and L. Conway, “Towards the principled engineering of knowledge”, AI magazine, vol. 3:3, pp.4─16, summer 1982.
[41] W. R. (Bert) Sutherland, “Management of industrial research : exploring the unknown technical future”, Perspective series 2008─7, Sun labs, July 2008.
[42] W. R. (Bert) Sutherland, “Faith, Funds, & Fate: prerequisites for the development and transfer of new technology”, Presentation at the celebration of the 40th anniversary of USC-ISI, Marina del Rey, California, April 26, 2012.
[43] L. Conway, “The VLSI archive”, Electronic design news, June 3, 2009. http://edagraffiti.com/?p=101
(Lynn Conway, Reminiscences of the VLSI revolution : how a series of failures triggered a paradigm shift in digital design, Fall 2012, IEEE solid-state circuits magazine. [2012])
____________________________________
Stoic at work : ancient wisdom to make your job a bit less annoying
by Annie Lawson
with illustrations by Oslo Davis
2023
text copyright by Annie Lawson 2023
p.140
Being able to switch between focus and daydreaming is an important skill, but it is curtailed by constant busyness. Engineering scarcity into our days can free up time for creative thought. [[ Engineering scarcity ? ]] Many important discoveries were made during downtime. Nikola Tesla had an insight into rotating magnetic fields while walking in Budapest. Albert Einstein liked to listen to Mozart during his breaks from intense thinking sessions.
Salvador Dalí believed he got a creativity boost during the early stage of sleep, when reality gets mixed up with fantasy. To use the technique, Dalí would hold an object, such as a spoon or a ball, while falling asleep in a chair. As he drifted off, the object would fall, make a noise and wake him up. Having spent a few moments on the brink of unconsciousness, he would be ready to start work.
Stoic at work : ancient wisdom to make your job a bit less annoying
by Annie Lawson
with illustrations by Oslo Davis
____________________________________
Oral History of Barry Boehm, part 2 of 2
Recorded February 20, 2018
[ software development ]
https://archive.computerhistory.org/resources/access/text/2022/07/102738733-05-01-acc.pdf
([ software is ... like ... the glue ])
Winston Royce, 1970, wrote a definitive paper for an electronics conference
described the waterfall model,
you really want to do some building it twice, so that you know roughly the directions you want to go.
prototyping is a form of risk reduction,
you want to be doing a combination of specification and prototypes.
process workshops
you were not going step-by-step, but you were going around a bunch of spiral circles.
the spiral model
got some better characteristics than the waterfall.
more risk reduction
good for big defense systems but heavyweight for business systems and the like.
agile method, an intial prototype.
“What you want to do is just interact with your customer to find out what they want and build them an initial prototype. What they don't like about the prototype, you do another sprint and fix that and say now what do you want.”
V-model, the waterfall tipped over.
Watts Humphrey
He was not particularly interested in how you build software, just whether your projections about cost and budget, scheduling and budget, could be trusted or not. His claim was, his motivation for the CMM, was just so that
the Defense Department knew which of those lying contractors could be believed, and which ones could not be believed.
If you got to Level 3 then you probably could make believable predictions. Then at that point, you're okay with Watts.
TRW
creating a new satellite system
the propulsion people
the structures people
the guidance and control people
the communication people
architectures
the software people
there are combinations of agile things that you want to do, and plan-driven things that you want to do.
Another book that I ended up co-authoring with a guy named
Rich [Richard] Turner was something called
Balancing Agility and Discipline.
It basically said that there are things where lockstep discipline is not a very good thing to do, but there are places where coordinating what you're doing is a good thing to do. Showing some examples of where the “Agile Manifesto”—they had a manifesto, that basically said things like, “Embracing change rather than following a plan.”
At one point at TRW, I was on a panel that was saying, “What were the causes of so many missiles getting launched from Vandenberg and then blowing up because of the software?”
In most cases, it was because people are responding to change over following a plan and saying, "We've got this fix that, we've got to do, or we've got this telemetry station that's moved and we've got to put a patch in the software. There's not enough time to do the regression testing and the configuration management and following the plan."
regression testing and the configuration management
And so, launched the rocket and boom, there it goes. Responding to change over following a plan may not be good in some situations.
sometimes the rocket blows up and sometimes it doesn't, and I want to be more predictable than that.
in fact for things that really matter, if your whole bank is going to rest on this or you what they have, and they want to be sure they can trust it.
Hubert Dreyfus, who wrote the book What Computers Can't Do, and showed all of the failed predictions that say, “In 1958, in 10 years, the computer will be the world's chess champion.” Well, they got there but not in ten years. I was getting sort of a balance of skepticism and enthusiasm.
Computer Systems Analysis Organization at RAND.
to apply system analyses to various challenges ── One of them was the Minuteman command and control system.
missiles that were in bunches in Montana, North Dakota, and various places.
People were worried about the fact that if the Russians targeted their missiles the right way they could disrupt the communication system so that the Minuteman wouldn't be able to launch.
finding that you needed a learning algorithm to determine if you had a system where parts of the system were going down, you needed some way to reroute things that weren't the shortest path until one of the thin nodes on the shortest path disappeared.
long, long ago for the first time, my boss said "Your job as a manager is to manage expectations. Never let people's expectations get out of the box, because if they get out of the box you can never win. If you do those incredible things people will say, ‘Well sure,’ but most likely you're not going to be able to do those things and people are going to get mad."
in 1955, “Pretty soon a computer's going to be the world's best chess player. Computers are going to automatically translate any language into any other language faster than anybody can even think the words.” Said all this crazy stuff, and it's just inevitable that at some point these promises are not going to be met and then there's a backlash and there's an AI winter.
1955
____________________________________
expense reducing technology
A dollar spent in the research lab for new product may be returned to the
company only after many years and much additional investment expanse produce
something that customers will buy in volume. The dollar spent in the research lab for some internal expense-reducing technology can be returned to the bottom line in the first year that the improvement is adopted in the company. The lab portfolio of activities must balance these two aspects of technology utilization.
Avoiding technical tarpits
In a similar vein, if the lab can clearly demonstrate what technology is not yet ready for prime time, its sponsor can avoid difficulty, expense and even
embarrassment. Several years ago, the digital transmission technology of
Asynchronous Transfer Mode (ATM) was the rage and widely touted for rapid
adoption to the desktop. A Sun Labs project tried to use ATM technology for our own building's internal network and found we could not make it work well
enough. This seeming failure helped change an incipient Sun campus network
plan from ATM to IP technology and avoid an expensive mistake.
a viable promising project ready to go with a champion is more concrete than ...
source:
[41] W. R. (Bert) Sutherland, “Management of industrial research : exploring the unknown technical future”, Perspective series 2008─7, Sun labs, July 2008.
(Lynn Conway, Reminiscences of the VLSI revolution : how a series of failures triggered a paradigm shift in digital design, Fall 2012, IEEE solid-state circuits magazine. [2012])
____________________________________
August 23, 2013
The Berkeley Par Lab:
Progress in the Parallel Computing Landscape
David Patterson, Dennis Gannon, and Michael Wrinn
Editors
• The Network of Workstations (NOW) project, 1993-1998, developed NOW-I and NOW-II, which were clusters of workstations that proved valuable in applications ranging from encryption to sorting. The Inktomi search engine was first built on NOW II, which led to a search engine startup company. Inktomi Inc. in turn demonstrated to the fledgling Internet industry the value of clusters of a large number of low-cost computers versus fewer more expensive high-end servers, which Google and others followed.
Project alumni became faculty at Berkeley, Harvard, Illinois, Princeton, Rutgers, Stanford, Texas, and Wisconsin. Tom Anderson, Eric Brewer, David Culler, and I led the NOW project. 7
• NOW was controversial in that it argued for building scalable large-scale computers from standard networks and computers as compared to distributed, cache-coherent, non-uniform memory access machines (CC-NUMA) as advocated by the Stanford DASH project and later embodied in the SGI Origin computers.
[7] A. Fox and D. Patterson. Engineering software as a service: An agile approach using cloud computing. Strawberry Canyon, 2013.
____________________________________
re-invention (Rogers, Everett M., Diffusion of innovation., 1. diffusion of innovation., 2. diffusion of innovations──study and teaching──history.)
Rogers, Everett M.
Diffusion of innovation.
rev. ed. of : Communication of innovations. 2nd ed. 1971.
bibliography
includes indexes
1. diffusion of innovation.
2. diffusion of innovations──study and teaching──history.
HM101.R57 1983
303.4'84
____________________________________
Mix it up ("high-low mix" concept)
The "high-low mix" concept, born out of the group of Air Force reformers sometimes called the "Fighter Mafia"—Cols. John Boyd, Thomas Christie, Franklin “Chuck” Spinney, and Pierre Sprey—sought to fix the death spiral of fighter aircraft development cost and complexity by augmenting a few expensive, high-capability aircraft built for air superiority with a larger number of less-complicated and less-expensive aircraft that could handle basic air defense, strike missions, and close air support.
In the 1970s, the "high" was the F-15 Eagle, and the "low" was the multirole F-16 Fighting Falcon. The Navy followed suit with the F-14 Tomcat as its "high" and the F/A-18 Hornet as a less-expensive "low."
To fix the death spiral of fighter aircraft development cost
• a few expensive, high-capability aircraft built for air superiority
• "high" was the F-15 Eagle
• a larger number of less-complicated and less-expensive aircraft that could handle basic air defense, strike missions, and close air support.
• "low" was the multirole F-16 Fighting Falcon
source:
https://www.textise.net/showText.aspx?strURL=https://arstechnica.com/tech-policy/2021/04/sitrep-is-the-f-35-now-officially-a-failure/
____________________________________
Steve Cross
One of my program managers was a Air Force major when I got there, and he got promoted to lieutenant colonel, but he introduced himself as saying, "I am the major cross that you're going to have to bear.”
He got some CMU [Carnegie Mellon University] and MIT people to come up with an AI constraint-based planning approach to solve transportation problems. This was in 1990 and '91, and they came up with a system that could do in four hours using constraint-based planning what it was taking the clunky transportation command software four days to do. Just about in 1991, we needed to get a half a million people off to the Middle East to fight the first Iraq war. And the transportation command said, “We're confiscating your Sun computers because we need your system to plan all of these things.”
Steve Cross got the Golden Nugget Award from the commander of the Air Force and went on from there.
Unfortunately, what I found was that a lot of organizations didn't want money added to their budget that they didn't control.
the two-star admiral who was the head of the Office of Naval Research.
"I get my money from the Chief of Naval Operations and I follow his priorities, and our big priority right now is corrosion. Our boats are getting corroded and we need more research in corrosion technology. And software, I can't really accept your software money. If I get more money I'm going to use it for corrosion."
DevOps [development operations]
in getting companies—the Defense Department to build things more like what the really advanced software companies like Amazon are doing. Amazon comes out with a new release of their software every 11 seconds, and there's this approach called DevOps [development operations] that fundamentally says that you organize your system in a way that many things can be going on in parallel, many things can be integrated in parallel, many things can be prioritized in terms of rushing testing and the like. So that the most important things that need to be in there, the next 11 seconds are going to be there, and then the flexible plan for what's going to happen in the next 11 seconds, and then the next hour,
Dean Leffingwell, who of built something called SAFe [Scaled Agile Framework], for “structured agile” or something.
Chalmers University in Sweden, Jan Bosch, who has made a success of doing this kind of thing, and has got a consortium of big European manufacturing companies, like BMW and Volkswagen and Saab and Volvo and Ericsson and the like.
Simon Ramo, who was the “R” in “TRW,” to help the company recruit more really bright software people because that was really a limiting feature on many of
the things that the company was trying to do.
to come up with a program where TRW would hire some really bright people, and give them two days a week to go to either USC or UCLA to get a master's degree. This would be a plum thing that really smart people would do.
UCLA and USC
For one, they were having a hard time hiring U.S. citizens.
Simon Ramo said, “We don't want to get super programmers out of this. What I found in our company the most valuable people we have are T-shape people. They're very deep in whatever their specialty is.
Maybe it's astrodynamics and maybe it's electronics, maybe it's communications, but maybe it's software. But the ones that are really helpful are the ones that know a little bit more about the other parts of it, about the management and the economics and all the other things.”
This was the basically true of most of the new hires that we would get as software engineers at TRW. It would take a year to get them to appreciate and understand what configuration management was all about, what requirements engineering was all about, how do you estimate costs of things and the like.
when I was 55 and saying, "I'm going to early retire or stay here for another ten years and get excused at 65, I think what I'd like to do is go to either UCLA or USC and take these I-shape bachelor's students in computer science and make T-shaped people out of them.” The software projects were a good example of that because they had to learn the requirements are things you negotiate. They aren't things that you just write up and follow.
basically they knew what configuration management was all about.
I think the main thing is that change is going to be constant.
“We want you to not just learn, we want you to learn how to learn.”
Software Management and Economics course
“You are the CTO [chief technical officer] of a 500 person software company and your chief executive officer is a concerned about AI, or DevOps, or Artificial Intelligence of various kinds and the like. What you need to do is to give him an analysis of how mature are these, and what are their strengths and what are their weaknesses, and what would we have to do to address these. And that you're going to get graded on how incisive your analysis is, plus the number of different ways that you learn about things. So you can't just go to Google and stop there. You should try to interview some people that are in companies, or are developing this kind of research and things like that. You should look at the proceedings of conferences and the ACM [Association for Computing Machinery]/IEEE [Institute of Electrical and Electronics Engineers] literature, and so the more sources that you do the better your grade is going to be.”
We've found that we have to learn
how to learn in running the course.
International Conference on Software Engineering (ICSE)
source:
Oral History of Barry Boehm, part 2 of 2
https://archive.computerhistory.org/resources/access/text/2022/07/102738733-05-01-acc.pdf
interviewd by :
David C. Brock
Lee Osterweil
Recorded February 20, 2018
____________________________________
Jon Gernter., The idea factory : the Bell Labs and the great age of American innovation, [2012]
p.51
In truth, large leaps forward in technology rarely have a precise point of origin. At the start, forces that precede an invention merely begin to align, often imperceptibly, as a group of people and ideas converge, until over the course of months or years (or decades) they gain clarity and momentum and the help of additional ideas and actors. Luck seems to matter, and so does timing, for it tends to be the case that the right answers, the right people, the right place ── perhaps all three ── require a serendipitous encounter with the right problem. And then ── sometimes ── a leap. Only in retrospect do such leaps look obvious. When Niels Bohr ── along with Einstein, the world's greatest physicist ── heard in 1938 that splitting a uranium atom could yield a tremendous burst of energy, he splapped his head and said, “Oh, what idiots we have been!”11
p.107
John Bardeen, the most careful of men, referred to his transitor work as a “discovery” of “transistor action”; he and Brattain had effectively observed in their experiment how a current applied to a slightly impure slice of germanium could hasten the movement of microscopic holes inside and thus amplify a signal.
p.107
a product like the transistor could ultimately fail for technical reasons (if it proved unreliable) or for manufacturing reasons (if it proved difficult to reproduce consistently or cheaply). Also, it might be the case that there was no market for a new device:
p.108
“But learning how to make them by the hundreds or thousands, and of sufficient uniformity to be interchangeable and reliable, was another problem.”31
p.108
Jack Morton
innovation was “a total process” of interrelated parts.
pp.108─109
“It is not just the discovery of new phenomena, nor the development of a new product or manufacturing technique, nor the creation of a new market. Rather, the process is all these things acting together in an integrated way toward a common industrial goal.”33
p.109
“reliability”, “reproducibility”, “designability”
pp.128─129
Rather, one might think of [information] in term of its ability to resolve uncertainty. Information provided a recipient with something that was not previously known, was not predict able, was not redundant. “We take the essence of information as the irreducible, fundamental underlying uncertainty that is removed by its receipt,” a Bell Labs executive named Bob Lucky explained some years later.30
(The idea factory : the Bell Labs and the great age of American innovation / Jon Gernter.
1. bell telephone laboratories──history──20th century.
2. telecommunication──united states──history──20th century.
3. technological innovations──united states──history──20th century.
4. creative ability──united states──history──20th century.
5. inventors──united states──history──20th century.
TK5102.3.U6G47 2012
384──dc23
384 Gernter
)
____________________________________
·‘’•─“”
<------------------------------------------------------------------------>
πόλλ' οἶδ' ἀλώπηξ,ἀλλ' ἐχῖνος ἓν μέγα πόλλ' οἶδ' ἀλώπηξ,ἀλλ' ἐχῖνος ἓν μέγα
____________________________________
*2 “This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold with the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional person should be sought.”
──From a Declaration of Principles jointly adopted by a Committee of the American Bar Association and a Committee of Publishers and Associations
(Ackoff's best : his classic writings on management, Russell L. Ackoff., © 1999, hardcover, John Wiley & Sons, Inc., p.139)
“This [copy & paste reference note] is designed to provide accurate and authoritative information in regard to the subject matter covered. It is [archive] with the understanding that the [researcher, investigator] is not engaged in rendering professional services. If professional advice or other
expert assistance is required, the services of a competent professional person should be sought.”
──From a Declaration of Principles jointly adopted by a Committee of the American Bar Association and a Committee of Publishers and Associations
--
No part of this publication may be reproduced, stored in, or introduced into a retrieval system, or transmitted, in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), without the prior permission of the publisher.
The W. Edwards Deming Institute. All rights reserved. Except as permitted under the United States copyright act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher.
All right reserved. No part of this book may be reproduced without written permission from the publisher, except by a reviewer who may quote brief passages or reproduce illustrations in a review with appropriate credits nor may any part of this book be reproduced, stored in a retrieval system, or transmitted in any form or by any means ── electronic, mechanical, photocopying, recording, or other ── without written permission from the publisher.
The information in this book is true and complete to the best of our knowlege. All recommendations are made without guarantee on the part of the author or Storey publishing. The author and publisher disclaim any liability in connection with the use of this information.
NOTICE: In accordance with Title 17 U.S.C., section 107, some material is provided without permission from the copyright owner, only for purposes of criticism, comment, scholarship and research under the "fair use" provisions of federal copyright laws. These materials may not be distributed further, except for "fair use," without permission of the copyright owner. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml
notice: Do not purchase this book with the hopes of curing cancer or any other chronic disease
We offer it for informative purposes to help cope with health situations and do not claim this book furnishes information as to an effective treatment or cure of the disease discussed ─ according to currently accepted medical opinion.
Although it is your right to adopt your own dietary and treating pattern, never the less suggestions offered in this book should not be applied to a specific individual except by his or her doctor who would be familiar with individual requirements and any possible complication. Never attempt a lengthy fast without competent professional supervision.
the home health handbook makes every effort to insure that its information is medically accurate and up-to-date. However, the information contained in this handbook is intended to complement, not substitute for, the advice of your own physician. Before embarking on any medical treatment or changing your present program, you should consult with your doctor, who can discuss your individual needs, symptoms and treatment.
All right reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording or by any information storage and retrieval system, without prior permission in writing from the publisher. The Australian copyright act 1968 (the Act) allows a maximum of one chapter or 10 per cent of this book, which ever is the greater, to be photocopied by any educational institution for its educational purposes provided that the educational institution (or body that administers it) has given a renumeration notice to the copyright agency (Australia) under the Act.
We acknowledge that we meet and work on the traditional lands of the Cammeraygal people of the Eora nation and pay our respects to their elders past, present and future.
No comments:
Post a Comment