Source: ENAEE-IEA, 2015: Best practice in accreditation of engineering programmes: an exemplar.
By: Dr. Eng. Daniel Hernandez Jimenez.
Head of the Accreditation Agency for Engineering and Architecture Programs.
Introduction:
Both the International Engineering Alliance (IEA) and the European Network for Engineering Education (ENAEE) jointly developed a model of best practices for the accreditation of engineering programs. The purpose of this model is to guide the accreditation agencies.
The best practices identified in the model are consistent with the applicable sections of the good practice guidelines published by INQAAHE (INQAAHE 2007), and the European standards and guidelines; applicable to the accreditation of a program (ENQA 2009). Also, the rules and procedures of the IEA (IEA 2013) and the framework of standards and guidelines (ENAEE 2015) of EUR-ACE® have been considered in the preparation of this model.
This synthesis shows the main aspects of the reference model, particularly the attributes of the graduates, the constitution, scope and government of the agency, the criteria for the accreditation of programs, the accreditation process: methods and means of delivery, and the ability of the agency to conduct accreditation activities.
The model does not cover methods or techniques for the implementation of best practices.
- Program results / attributes of the graduates.
These are evaluable learning outcomes, which describe or exemplify the knowledge, the skills and attitudes expected of a graduate of an accredited program, which provide the educational foundations for a purpose, including practice in a specific field of the engineering profession.
In this regard, knowledge is understood as: the facts and concepts that are known and understood. Skills:the skills to manage and apply knowledge. Attitudes: the purposes that should lead knowledge and skills.
- Best practices: definition and use.
A best practice is a methodology that, through experience and reflection, reliably leads to a desired result, superior to those obtained with other means. They provide a reference point for an accreditation system in the evaluation of programs.
A best practice can evolve to become the best par excellence, as new developments are discovered.
They are addressed to the accreditation agencies, not to the programs. Also to organizations that recognize accreditations as “substantially equivalent”: ENAEE, IEA.
It is based on the principle that: Agencies are not involved in the design and offer of training programs.
The elements of the best practices can be adapted in the policies, processes and criteria of the accreditation agencies and recognition bodies.
The elements of the best practices are those that have been found to guarantee the best functioning of the accreditation systems.
- Agency constitution, scope and government.
The agency must be legally constituted in its jurisdiction or be a duly constituted bureau or committee of a legal entity and that, in any case, has an adequate property and government structure.
The agency is recognized within its jurisdiction, as the exclusive or principal authority for the accreditation of engineering programs.
The agency operates with consistency and transparency, in a defined relationship with any national educational regulation, such as a qualification framework, or a quality assurance system.
The agency has a clear responsibility within its mission for the accreditation of engineering education programs, whose main purpose is to provide the educational base for independent practice in a defined engineering profession.
The agency is independent and acts autonomously with respect to accreditation. It has full responsibility for its operations and the accreditation decisions must be made without the influence of third parties.
The agency considers for accreditation only programs offered by providers that have legal authority to operate and grant degrees or diplomas.
The geographical limits of the accreditation activity, if any, should be defined, indicating the differences in standards, processes and levels of recognition of the programs if they are different in different territories.
The accreditation and the agency have the support and have well established relations with the key actors in the engineering academy and the industry communities.
The agency makes accreditation decisions, based on the judgment of peers.
The agency, its administrative staff and expert peers, observe the principles of governance and act with professionalism.
Management, evaluation and accreditation decisions must be made in a balanced manner, based on inputs provided by industry and education peers, drawn mainly from the jurisdiction where the engineering programs operate.
The providers of education programs, although they are key actors, do not have power to control the rules, policies and decisions of accreditation of the agency.
Functions and powers are clearly assigned to the council, committees or other structures responsible for the global governance, setting standards, accreditation criteria and policies, as well as evaluating programs and making accreditation decisions.
An agency may delegate functions related to the accreditation of programs to designated agents. In any case, the agency maintains responsibility for the quality of the accreditation process, the standards applied and their consistency. The delegated agents are also subject to the best practices.
- Criteria for accreditation
If the agency has mentoring procedures to assist the accreditation applicants, these activities are clearly separated from the accreditation activities.
The agency has available to the public, the criteria for the evaluation and accreditation of programs.
The Agency develops and reviews the standards, criteria and policies through a process that considers as input the comments of the engineering peers and the public, including the contributions of relevant engineering stakeholders.
The criteria to be met by the accredited programs are the following:
- The purpose of the program includes providing the engineering education foundation for an established profession.
- The rules of the agency, define that the results that must show the graduates of a program:
- Are consistent with the purpose of the program;
- Are consistent with an international standard such as, for example, the IEA model of graduates’ attributes or the results of the EUR-ACE® program.
- The entry requirements of the students are defined by the program provider in a consistent way with the demands of the curriculum, the preparation of the students, and the expected progression.
- The design of the program of the provider is coherent and consistent with the objective of the program, the established entry requirements, and the results of the program to be fulfilled.
- The student’s progress requirements are consistent with the entry level, the curriculum and the learning outcomes of the program to be achieved.
- The evaluation of the students is an adequate combination of formative evaluation and summative evaluation, consistent with the expected progression of the students and the learning results that are sought.
- There is a process to ensure that the individual assessment of students meet the required standard and is consistent, objective and fair as well.
- The learning and teaching environment is appropriate for the objectives of the program, its structure, the training of the students and the established results.
- There is an adequate environment with the necessary resources to offer the program including:
- Appropriate leadership for the program.
- The program professors have an adequate combination of academic, professional and experience qualifications.
III. Adequate physical and financial resources to support the program.
- Adequate planning and execution capacity.
- The provider carries out reviews and actions for the continuous improvement of the program, with the participation of students, employers, graduates and other interested parties.
The accreditation criteria is defined in such a way that it gives the program provider freedom to design and execute programs to meet the standard of graduates’ attributes.
The accreditation process requires the program provider to report on the quality resulting from the program and its execution, the achievement of the attributes of the graduates and the continuous improvement of the program.
- The accreditation process: methods and means of delivery
The program evaluation and accreditation decision making are based on appropriate, coherent and fair procedures.
If the accreditation process is done in conjunction with another quality assessment or control process, there must be clear distinctions between the two sets of criteria and decision making.
The evaluators, the members of the accreditation decision-making bodies and the administrators, always operate in accordance with high standards of professionalism, ethics, impartiality and objectivity.
The accreditation process is documented, is self-consistent and supports the evaluation of the program based on the fundamental criteria for accreditation.
The accreditation system must produce complete information to all interested parties, while maintaining an adequate balance of transparency and confidentiality, in the process of accreditation of individual programs.
Within the process of accreditation of a program:
- The agency, evaluators, observers, decision makers and administrative staff maintain confidentiality about the information obtained.
- At the same time, the evaluators are open with the provider; in particular, if possible adverse outcomes are anticipated, they grant an opportunity to provide additional information.
The accreditation system is completely transparent, as shown by:
- Officially, written policies, standards, criteria and procedures are available to providers and to the public.
- Those involved in the accreditation process have access to knowledge and skills in matters related to engineering accreditation, engineering education, training experiences for students and engineering practice.
The agency has a policy on observers, including confidentiality requirements, restriction of influence on the process and participation protocols.
A process to appeal adverse accreditation decisions is available, involving only people with no prior participation in the decision that is being appealed and with no conflict of interest.
A clear policy on conflicts of interest exists for all those involved in the accreditation process, including visiting teams, accreditation decision makers and policymakers.
There are practices to ensure that there is no conflict of interest at any stage of the accreditation process, including the selection of program evaluators and during the accreditation decision making.
Accreditation decisions are made for each individual program identified according to the provider’s rules, the degree certificate and the academic record.
When appropriate, the evaluation process can consider groups of related programs together.
In cases where the program is offered through multiple channels, the path followed by a student is disclosed in the certificate of degree or academic record.
The evaluation of the programs is carried out by peer evaluators with the disciplinary knowledge of the programs under evaluation and a balance between academics and engineering professionals.
Where the practice is for a student or students to be part of the visiting team, the following applies:
- The student or students contribute to the evaluation of programs in areas where they are competent, including:
- Meeting with a representative sample of students of the programs being evaluated and student leadership;
- Participate in the evaluation from the student´s perspective of evaluation services, teaching and learning, library support, safety aspects as taught and practiced, student understanding of the educational objectives of the program.
III. Participate in the evaluation of learning conditions as they are perceived by the students: programming, pedagogical methods, workload, etc …
- To be eligible for inclusion in an evaluation team, the student must:
- Be enrolled in a licenciature or master’s degree in the engineering program.
- Have demonstrated leadership among students.
III. Not having a disciplinary action in his/her academic record.
- Have attended the training as required by the accreditation agency.
The evaluation process includes a visit to the program provider’s facilities.
The accreditation process includes periodic reevaluation to maintain accreditation and a follow-up process when dictated by recommendations and decisions.
The accreditation process requires:
- A self-evaluation or self-study report completed by the program provider before the visit. It is delivered through a structure and a format specified by the agency, which must gather information that demonstrates how the program meets the accreditation criteria. The responsibility for such demonstration rests on the program provider´s side.
- A description of the curriculum is part of the self-study report and provides the complete information regarding all the modules in the program.
- Documentation must be provided to the evaluators in time for proper preparation for the visit.
- Clearly specified evidence must be available on the site during the visit.
The agenda of the visit gives time and opportunity for the evaluators to:
- Gather, verify and evaluate all necessary information, including the evaluation of the evidence of student achievement of the attributes of the graduate.
- Interview the main actors of the program (administration, teachers, students, staff, employers).
- Examine and evaluate available facilities (including computer labs, laboratories, etc.).
- Carry out a private discussion, for reflection and refinement of the evaluation as a product of the visit.
Accreditation decisions:
- It is an evaluation based on criteria, within the framework of an exercise of judgment by pairs.
- Makes a judgment of the sustainability of the program.
- Has a clearly defined set of allowed accreditation processes and decisions, including assessments at predetermined regular intervals and provisional evaluations under defined conditions.
- It has a rational approach to deal with programs that do not comply, which encourages and verifies the improvement of quality.
- It is receptive to innovation in engineering technologies and teaching methods and does not inhibit the introduction of new topics and forms of teaching.
- It has a method of dealing with new programs, with extensively revised programs with material changes during the period in which the program is accredited, and with terminal programs.
The Agency follows protocols defined for the reports. In particular:
- The visit reports provide sufficient detail for the accreditation council (or equivalent) to make informed decisions about the accreditation or non-accreditation of programs, or to impose conditions.
- The reports use standardized forms to record decisions and recommendations. Key words are defined, such as: deficiency, weakness, concern, compliance, recommendation.
- Evaluation templates or questionnaires can be used, team results are clearly recorded, their recommendations and decisions are based on criteria and evidence and are developed in the form of a report.
- The agency provides a written report to the institution that clearly distinguishes between the actions required for the provider to achieve or maintain the accreditation and the recommended measures for the improvement of the academic program.
- The process allows the right to reply on factual issues, by a relevant official of the provider, for example, Dean or program manager, before making the recommendation or decision of accreditation.
The agency publishes or makes available to the public a list or database of accredited programs, which clearly identifies each program on which an accreditation decision has been made and the period of validity thereof.
The policy of the agency may require the publication of the logic of its decisions or others, subject to the limitations derived from confidentiality and other relevant considerations.
In the actual operation of the accreditation system, the evaluator and decision maker and administrative staff, must ensure that:
- Accreditation visits are carried out in accordance with the published accreditation policies and procedures.
- Standards and criteria are applied in a coherent and fair manner, from institution to institution, from program to program, and from year to year.
- The accreditation system and the way it operates is robust; unusual circumstances are handled in a sensitive manner and tough decisions are made in a manner that is beneficial to the engineering community in the long term.
- Substantial changes in accreditation system standards, criteria, policies or procedures are handled realistically for all interested parties.
- Capacity of the agency to conduct accreditation activities
Accreditation of engineering programs requires resources to implement at the beginning and maintain an accreditation system. The development of education programs must be maintained over an extended period, and the cyclic process of accreditation must also be sustained indefinitely.
The providers and the agency require continuous improvement. The best practices therefore require that the agency has the capacity to develop initially (if applicable), operate and develop accreditation activities on a permanent basis.
Essential components are:
The agency has sufficient funds and the funding prospects are sustainable to support an effective and sustainable accreditation system.
The agency has sufficient staff, duly trained to administer and effectively operate the accreditation process.
An adequate number of qualified engineering peers, with academic background and that of the industry is available for appointment to the accreditation board and its substructures.
A body of experienced evaluators is available in all disciplines in which programs are presented for accreditation and to be visiting team leaders as well.
An effective process is applied for the recruitment, selection, training and evaluation of program evaluators.
The corresponding eligibility criteria is applied in the selection of evaluators.
There is an effective training process for the evaluators and it is supported by written teaching material.
The agency performs comparative analyzes of accreditation, its criteria and processes. It also conducts periodic self-evaluations to improve its standards, criteria, policies and procedures.
The methods may include the use of international evaluators and observation of the agency´s processes.
The procedures of the agency ensure that its standards and working methods are reviewed at regular intervals, are subject to external scrutiny and are updated when necessary.
- Conclusions
Both the providers of university education aimed at the training of future professionals in Engineering and Architecture, as well as the accreditation agencies, must assume a commitment to improvement to effectively carry out the mission demanded by society.
The guide of the best practices proposed jointly by the IEA and the ENAEE, are a valuable instrument to promote the continuous improvement of the accreditation agencies and influence in a positive way on the training programs.
The guidelines are proposed as recommendations and cover the desirable aspects referred to the constitution of the agencies, the criteria, processes and training.