The questionnaire shown below was sent in advance to all the
organizations visited in this study, except for Nikkei AI.
Section A focuses on industrial applications, and hence is relevant to
the twelve industrial sites only. Section B contains questions about
advanced research and was potentially relevant to all the recipients.
The questions in Sections C and E were directed specifically to ICOT
personnel. Section D was prepared specifically for the EDR project.
The questionnaire is quite long and detailed (particularly Section
A), and we were not sure if any of our hosts would take the time to
prepare written answers to the questions or merely use them as talking
points at the meetings. We were surprised and delighted at the level of
effort expended by most of the organizations in preparing written
answers. In particular, Section A was completed by all twelve
industrial sites. Section B was addressed by a few industrial sites,
and served as a guide for discussion at the universities and national
projects. ICOT did not prepare written responses to Section C and E,
but the topics were discussed during the visit; the same situation
applies to the EDR project.
A. INDUSTRIAL APPLICATIONS
The following questions are for organizations that have developed
and are using expert systems.
- Approximately how many expert systems has your company developed,
or is developing?
- for internal use?
- for external use by customers?
- Please categorize the above according to development status (give
- in routine operation
- now in field test
- in prototype stage
- prototype still in development
- Please state approximately how many applications you have in each
of the following task areas:
- advisory, or "help" systems
- process control
- planning and/or scheduling
- diagnosis and/or troubleshooting
- regulatory compliance
- management or business integration aids
- areas of manufacturing not included in any of the above
- software development and management
- intelligent interfaces to existing software systems
- What is the current most successful expert system that your company
developed for its own use or for a customer? How do you measure its
- For each task area where you have one or more applications, select
a representative system (including the most successful described
above), and provide the following information:
- what is the specific task?
- how large is the knowledge base (number of frames, number of rules,
lines of code, as appropriate)?
- was a commercial development tool or framework used to build the
application? If so, which one? If not, was an internally developed tool
- is the application a stand-alone system, or is it integrated with
conventional data processing systems?
- what programming language was used (if any)?
- what kind of hardware platform (personal computer, workstation,
minicomputer, mainframe, client-server system, distributed or parallel
- what was the size and composition of the team that designed and
built the system?
- how long was the development time?
- what was the development cost?
- who developed the system (domain experts, professional programmers,
professional knowledge engineers)?
- who are the users?
- who maintains the system, and how large is the maintenance
- what kind of payback has been realized from this application (e.g.,
cost reduction, quality improvement, speed of problem solving or
- in economic terms, what has been the return per year?
- what was the payback time, i.e., the number of months or years to
pay back development costs?
- does this application cooperate with or depend on other expert
systems (i.e., do they contribute to each other's operation in some
way) and if so is there a synergy between the applications?
- what problems were encountered, if any, in integrating the system
into the corporate environment, and how were these problems
- Has your company implemented any applications that require multiple
sources of expertise? If so, was the expertise combined in a single
system or through the coordination of multiple systems?
- Do you have any applications written partly using AI tools and
partly using standard high level languages (COBOL, FORTRAN, PL/1,
- How fully integrated with your operating system are your AI tools?
E.g., can you call any access methods, procedural higher level language
routines, library subroutines from your AI tool (and vice versa)?
- What training programs for expert system development work are in
place in your company?
- Has your company developed a methodology for developing expert
systems? If so, what is the methodology?
- In your company, how are expert systems
- selected (i.e., what criteria are used)?
- developed (i.e., by data processing group, by a knowledge
engineering group, by outside contractor)?
- inserted into the company's operational activity?
- integrated with conventional data processing activities?
- Are there systematic review and redesign cycles for expert system
- What percentage of projects that are started get as far as
completing a prototype system?
- What percentage of projects get from the prototype stage to an
- Have there been any expert system projects that were unsuccessful?
If so, what were the reasons for the lack of success? Was the problem
caught early or late in the development cycle?
- Given your existing AI tools, how large a project would your
organization undertake (e.g., in terms of number of rules or number of
person-years devoted to the project)?
- What new expert system applications are planned by your company?
- What will be the progress of expert systems in your company? For
example, will process control applications use in-the-loop control?
- What are the main problems with present technology? What kinds of
applications that are hard to build now would you like to see made
easier? Do you see any need for advanced techniques (e.g., model-based
reasoning, machine learning) in current or planned expert system
applications? Do you see resistance to the use of those technologies?
- What other advanced computing technologies does your company use
(not necessarily expert systems)?
B. ADVANCED KNOWLEDGE-BASED SYSTEMS RESEARCH
- Are any efforts being made (besides EDR) to build the technology
for large knowledge bases? Are there experiments in large knowledge
bases? Is there a software or communication infrastructure that has
- What is the research being done experimentally or theoretically on
model-based reasoning? Modeling of devices and reasoning about their
behavior? Are any languages or systems being developed to assist with
- What are the major projects in machine learning (using
knowledge-based techniques, not neural networks)?
- Have you developed any new techniques to handle problems of
reasoning with uncertain knowledge?
- If you think of advanced knowledge-based systems research as
involved in "inventing the second generation" of knowledge-based
systems, then what are the most important dimensions of this "second
generation?" That is, what is it most important for the researchers to
- What do you think are the important features that should be
included in second generation development tools or frameworks?
- What is the interest among computer science and engineering
students in knowledge-based systems research? How many Ph.D. students
do you have working with you, or are working on knowledge-based systems
in your department?
- Are you working on any advanced truth maintenance systems?
- Are you applying knowledge-based systems to natural language
understanding? To problems of education, at any level?
- Are you doing any work in case-based reasoning research? What
efforts in Japan in CBR are most worth studying?
- Are you working on general problems of knowledge representation,
for example by inventing new concepts of KR, or new languages for KR,
or new systems for KR?
- What are your sources of support for the research? From Ministry of
Education? From other government agencies? Does any support come from
- What do you consider to be the major technical successes on the
Personal Sequential Inference (PSI) machine? How successful was the
effort to commercialize the PSI machine?
- What is the current work of the various ICOT laboratories?
- What performance is expected from the Parallel Inference Machine
(PIM) to be completed in 1992?
- Briefly described, what is the current PIM architecture -- that is,
- what are the activities at a node?
- how is the communication between nodes handled?
- what are the high-level parallel algorithms that organize the
logical problem solving?
- What is the current state of the operating system for the PIM?
- What is the current state of ICOT's PROLOG work?
- What applications are being done at ICOT? Are there expert system
applications? What organizations outside of ICOT are using
ICOT-developed technology to build expert system applications? What is
the best application done with ICOT-developed technology?
- What are the major advances that have been made by ICOT in the
Natural Language Understanding area?
- Can you quantify the role that ICOT has played in spreading
knowledge of AI and logic programming among Japanese company engineers
- Assuming that there will be a second period of government funding
for ICOT, what are the plans? How long? How many people at ICOT? What
research will be undertaken? What level of government funding has been
- Of all the things that ICOT has done in ten years, what stands out
as the most satisfying to the Director and the staff? What was the most
disappointing experience of the ten-year period?
- Several Japanese companies (and a few American companies) also
explored novel architectures for LISP. Was there any relationship
between this work and the work at ICOT? If so, how did the efforts
influence one another?
- PROLOG and other logic-programming formalisms have not captured the
same attention in the U.S. AI community as they have in Japan. Are you
satisfied with your choice of logic-programming as the basic framework?
Do you think that you should have adopted more of the functional
programming style of the LISP-like languages?
- While PSI and PIM were being developed, the RISC revolution took
place. To what extent did this affect your work? Would you have adopted
more of a RISC-like architecture if your work had started later?
D. ELECTRONIC DICTIONARY RESEARCH PROJECT
- What is the current state of the EDR project? In particular, what
is the current state of completion of the semantic dictionary?
- Please give us some current details about implementation. What
language is being used to handle representation? What set of concept
primitives are you using?
- As the knowledge base gets larger and larger, what problems of
scale-up are you encountering, and how are you handling them?
- Are the dictionaries (including semantic dictionary) in trial use
in any of the companies? How is the use done?
- Are you going to use ICOT's PIM for running the dictionaries for
natural language understanding projects?
- How big is the current staff of EDR? What are the plans for EDR?
Will it grow? Shrink? What new projects will it undertake?
E. REAL WORLD COMPUTING PROGRAM
- What research efforts will RWCP undertake in the area of AI or,
more specifically, in the area of knowledge-based systems?
- For these AI-related or KBS-related projects, how much government
funding is planned? Over what length of time? What companies are
involved? Will there be a project institute? What is ETL's role in
- Will these AI-related research projects focus on particular
application domains (perhaps as testbeds), and if so what are they?
- What role will parallel computing play in these AI-related
projects? What kind of parallel computing is envisioned?
Published: May 1993; WTEC