CELT Newsletter, Spring 2007

Abstract

The Hong Kong University of Science and Technology Center for Enhanced Learning and Teaching Spring 2007 Shifting the Program Design Focus In Preparation for the 4-Year Degree CU Fifth Round Awarded Projects hove begun! 10-11 Tech Comp 2006 Winter 12 Recent Events Highlights 13 Shifting the Program Design Focus: In Preparation for the 4-Year Degree HKUST is on its way to the implementation of a four-year degree. One major implication of this is that some thought is required in planning for the curriculum, courses, and assessments. In this edition, we will look at 'Intended Learning Outcomes', including what they are in general terms and in a HKUST context, how assessments can be aligned with intended learning outcomes for the benefit of student learning and development. We will start by going through the types of intended learning outcomes and the desired levels to be achieved by students over a 4-year program. This will be followed by four program document examples from different universities. Intended Learning Outcomes The Accreditation Board for Engineering and Technology (ABET) defines the term 'outcomes' as follows: 'Program outcomes are statements that describe what students are expected to know and be able to do by the time of graduation. These relate to the skills, knowledge, and behaviors that students acquire in their matriculation through the program'. These are the three types of intended learning outcomes and Disciplinary Knowledge Create the levels expected to D Evaluate (e.g. factual, conceptual, Analyse be achieved by students procedural knowledge) over a program. Understand Remember Generic Skills (e.g. team working, Mastery communication) Production Conformation Simulation Perception Attitudes (e.g. ethical behavior, Behaving social responsibility, Believing professionalism) Valuing Responding Receiving Now let us take a closer look at the 'Disciplinary Knowledge' part by using Bloom's Taxonomy of Cognitive Outcomes and see how the progression can go within a four-year time be. Using Anderson et al's Taxonomy of Cognitive Outcomes as a Framework for the Disciplinary Knowledge of &Year Program Design In Anderson et al's Taxonomy of Cognitive Outcomes, different levels of learning are identified. They move from the 'simplest to the most advanced'. The levels and the related cognitive processes are listed in the table below: Levels of Learning: Cognitive Process: Level 6: Creating Put elements together to form a coherent or functional whole; reorganize elements into a new pattern or structure Level 5: Evaluating Make judgments based on criteria and standards Level 4: Analysing Break material into its constituent parts and determine how the parts relate to one another and to an overall structure or purpose Level 1: Remembering Retrieve relevant knowledge from long-term memory The levels of learning and related cognitive processes> Over the 4-year degree program there should be a broad development trend from lower to higher cognitive outcomes which would look like this: High-level cognitive outcomes Applying @ 3 3 2 3 @ -3 3 3 3 Applying Low-level cognitive Understanding & !& & & b C G C 6 dU Understanding outcomes Remembering G & & L & C C C Remembering Year 1 Year 2 Year 3 Year 4 In the early years, lower level cognitive outcomes, i.e. 'Remembering' and 'Understanding' are given stronger emphasis. The level moves upwards as the year moves on. Higher level outcomes like 'Evaluating' and 'Creating' would have more emphasis in later years. It is a curriculum team's responsibility to ensure this development over the program and to make sure the four types of intended learning outcomes are covered. In the process of writing learning outcomes, the curriculum team would use associated action verbs for different levels of learning. Some verbs are listed in the table for consideration. Level 6: Creating Hypothesise, Design, Construct, Plan, Invent, Devise Level 4: Analysing Differentiate, Discriminate, Distinguish Level 2: Understanding Level 1 : Remembering Recognise, Identify, Define, Retrieve, Recall, Record Level appropriate to different levels of Anderson et ai's Taxonomy - ABC LIVE at HKUST Academic Excellence At HKUST, program outcomes need to be 8 road-based education aligned with HKUST's seven graduate e attributes - called ABC LIVE: Competencies and capacity building Leadership and teamwork International outlook Vision and an orientation to the future V. Ethical standards and compassion Shifting the Program Design Focus: In Preparation for the 4-Year Degree Examples of Program Documents In this part, four assessment documents from different universities for Science, Engineering, Business, and Social Science and Humanities will be discussed. You will see the extracted 'Learning outcomes' and 'Assessment' in each example. Annotations include the type of learning outcomes and how the assessments align with those learning outcomes. Example for Science Name of Institute: University of Minnesota Department: Biology U RL: http:/hww.mrs.umn.edu/commiitees/asl/biology shtml The Biology Department of University of Minnesota uses summative, formative and sustainable assessments to measure students' attainment of learning in different areas. There is a very strong alignment between the intended learning outcomes and the assessments. Students Learning Outcomes: 1 Disciplinary Knowledge 1. Students will be able to summarize the major ideas of biology. Accurately and concisely and to apply these ideas to a variety of organisms and situations, Generic Skills 2. Students will be able to perform successfully a variety of field studies and laboratory experiments 3. Students will be able to prepare and present clear, cogent, 1 and appropriate talks and papers dealing with a number of, This summative assessment aligns with biological topics: a - -3 Learning Outcome Methods of Assessment: This formative assessment aligns with Learning Outcomes 1 , 2 and 3. 1 Unit or course examinations, term papers ( This formative assessment aligns with 287 Learning Outcomes 2 and 3. 2. Standardized examinations 3. Oral presentations This sustainable assessment task aligns 4. Written lab report and/or poster sessions with Learning Outcomes 1,2 and 3. Faculty-supervised research project and other research- This formative assessment aligns with Learning Outcome 2. related activities 7. Portfolio includes lab report and related materials, along with I This sustainable assessment task aligns summaries of any additional research experiences with Learning Outcomes 1,2 and 3:

Shifting of Program Design Focus: In Preparation for the 4-Year Degree Example for Engineering Name of Institute: The University of Texas Pan American College: Science and Engineering Degree Program: B.S. in Mechanical Engineering URL: http://www.ie.utpa. edu/SLOutcomes~SpringO6/CoS~BSMechanicalEnggSpnngO6.pdf This comes from College of Science and Engineering Student Learning Outcomes Plans and Assessment Reports. The Learning Outcomes are in two parts: 1. School Level: Student Learning Outcomes 2. Department Level: Program Specific Outcomes for Mechanical Engineering In some programs, the alignment between Learning Outcomes and Assessment might not be as clearly mapped as in this one. : Students Learning Outcomes (for Science and Engineering College) (p.1) Disciplinary Knowledge It will be demonstrated that the student: I Disciplinary Skills 1. is able to use knowledge of mathematics, basic sciences and engineering #' Generic Skills to analyze (identify, formulate, and solve) problems in mechanical engineering. 3. is able to design mechanical devices, systems or processes that meet more specific, e.g. mabible 1 to explain'. 5. is able to communicate ideas effectively in graphical, oral and in written, media Attitudes professional responsibility of an engineer and how engineering solutions impact safety, economics, ethics, politics, societal, These summative cultural and contemporary issues. assessments align with Learning Outcome I. Means of Assessment that Aligns with the above Student Learning Outcomes (p.1) These summative assessments align with Home work; Tests; Pre-test Examination at the end of senior year; Senior , design project; Performance on the Fundamentals of Engineering ; Examination (FE exam). These summative assessments align with 3. Senior design Project; Building a model; Oral Presentation and written Learning Outcome 5. 5. Written and oral Presentation of senior design project B' These summative 6. Senior design project; Engineering Ethics Program Specific Outcomes for Mechanical Engineering (p.4) It will be demonstrated that the student: 9 S1. has the knowledge 7 A s of chemistry Disciplinary Skills with depth in at least one C has the ability to apply advanced mathematics to problems, Disciplinary Skills 7 involving thermal and mechanical systems S3. has the ability to a pity statistics and linear algebra to problems These formative/ summative involving thermal and mechanical systems assessments align with Learning Outcome Means of Assessment (p.4) These formative/ summative - a assessments align with Learning Shifting of Program Design Focus: In Preparation for the 4-Year Degree Example for Business Name of Institute: Eastern Oregon University School of Education and Business: Business Division URL: http://www.eou.eddctl/documents/BUSINESS%20ASSESSMENTO/o20PLAN. doc The alignment between intended learning outcomes and assessments is strongest for intended learning outcomes 1 to 4 and weakest for 5. Outcomes of the Business Administration Degree Program (p.3) 1. Graduates of the Business Administration degree program will Disciplinary Knowledge. possess an integrated knowledge of the functions and systems 'Possess' may be difficult to measure. Cognitive processes of business such as summarize or 'infer' I could be considered. 2. Graduates of the Business Administration degree program will have developed strong interpersonal and communication skills -I necessary to succeed in the business enviro0nment. +, Generic Skills 3. Graduates of the Business Administration degree program will have strong technological skills including the use of the Internet, Skills. Depending on what is word processing, spreadsheets, and presentation software. - - - may be required e.g. subject specific spreadsheets 4. Graduates of the Business Administration degree program will be able to demonstrate the capability of applying business concepts to real world projects

Is this part of an assessment?

5. Graduates of the Business Administration degree program will Can 'demonstrate the capability perform effectively and efficiently in business managerial positions, of' be omitted? 6. Faculty, administrators, and staff of the business division create an environment that fosters excellence in learning.

7. The Business Administration Degree Program provides the These are not intended learning necessary education and experiences that enable graduates to outcomes for students. be effective leaders in a global business environment. Student Outcomes Assessment (p.4) 1. National Exam- Major Fields Test. Used to assess fundamental knowledge of business concepts. Scores will be compared to the national average. This test will normally be administered in BA This summative assessment 498 Business Policy and Strategy. aligns with intended Learning Outcome 1. 2. Integrated Case Study. Used to assess students' ability to integrate knowledge and use it to make decisions and to assess \ ( This summative assessment effectiveness as a member of a team. (Scoring rubric Appendix). aligns with intended Learning The case study will be required for BA498 Business Policy and Outcomes 1 and 2. Strategy. 2 3. Senior Project. To assess students' ability to apply business concepts and the capability of planning and organizing a major task as well as to assess students0 oral and written communication skills and team skills. (Scoring rubrics Appendix). To take place in BA490 Senior Project. 4. Technology Portfolio. To assess whether students' have advanced technological skills. Assessment will take place in BA 325 Information Management. (Scoring rubric Appendix).

This summative assessment aligns with intended Learning Outcomes 2 and 4.

Shifting of Program Design Focus: In Preparation for the 4-Year Degree Example for Humanities and Social Science Name of Institute: University of Maryland University College Department: Communication, Arts, and Humanities Program: Baccalaureate Degree Program in Humanities URL: http : /m .umuc.edu/outcomes/susplans/HUMS.pdf Overall, the alignment between intended learning outcomes and assessments is quite strong for all outcomes except for 1 and 14. Program Outcomes (p.7): Upon completion of the Baccalaureate Degree Program in Humanities, graduates will be able to: 1. Use effective oral communication skills to present ideas in both : public and interpersonal settings' , Create written communication appropriate for the purpose and E which meets standards of style and grammatical correctness. ; Write analytical, critical, research-based prose that addresses 5 interdisciplinary issues. Disciplinary Knowledge, Disciplinary Skills Evaluate technological concepts related to computers and , n / components of information systems. Use online resources to locate scholarly articles and other ; materials for interdisciplinary research. 6. : Address recognized research needs by retrieving, evaluating, I a and using information appropriately.7. ! Use primary and secondary sources to conduct interdisciplinary : Disciplinary Knowledge, Research effectively. Disciplinary Skills 8. Apply mathematical and numerical reasoning skills. : Disciplinary Knowledge, Disciplinary Skills Exhibit quantitative skills necessary for analysis and synthesis of quantifiable human experience Disciplinary Knowledge, Analyze issues across traditional discipline boundaries. 11. Identify key concepts and principles of natural sciences. 12. Analyze the influence of science on the development of contemporary Humanities perspectives Disciplinary Knowledge, Disciplinary Skills 13. Demonstrate knowledge of and appreciation for the traditions and major ideas within the sphere of Asian economic, political, cultural, and historical developments. 14. Integrate knowledge of other cultures to respond appropriately part of an assessment? Consider to use alternative to a varietuy of cross-cultural situations I words such as 'explain' or 'describe'. 15. Evaluate current events based on chronological knowledge of various historical events, ideas, and individuals. Disciplinary Knowledge 16. Use knowledge of other cultures, politics, ethics, and human rights to positively impact the community, work, or physical environment Methods of Assessment (p. 8-11) Disciplinary Knowledge, Disciplinary Skills, Generic Skills, 2. Examinations (Course/Chapter) This summative assessment 1 3. Essay writing task aligns with Learning Outcome 2. 1

This formative assessment aligns with Learning Outcomes 3, 5, 7, 9, 10, 12, 13 and 16. Projects have Begun! CLI Fifth Round Awarded Projects have Begun This is the fifth year of the Continuous Learning and Improvement (CLI) through teaching innovation project. The focus of the fifth round CLI projects is on the improvement of student learning. Research (Biggs 2003) in recent years has shown that to bring about more effective learning, students should be encouraged to adopt a deep learning approach, which can be achieved by such methods as engaging students in authentic learning experience, adopting student-oriented approach in teaching, using appropriate forms of assessments, and setting the students' workload at an appropriate level. In order to evaluate the impact of their teaching innovations on student learning, CLI project leaders are encouraged to make systematic effort to collect solid evidence of improvement in students' performance. Twelve CLI projects have been awarded: one program-level, six course-level and five adaptations. Here's a quick glimpse of the sub-projects and their main objective: Pilot Courses for Science General Education in the Four-Year University Curriculum, Michael Wong (PHYS) To develop and offer new joint department General Education (GEE) based on inter-disciplinary themes. Spreadsheet Engineering -The Application of Excel's Macro Programming Language, (Visual Basic for Applications) to Core Courses within Chemical Engineering, John Barford (CENG) To provide structured teaching of Excel - Visual Basic Application (VBA) in an introductory first year course and application of VBA throughout major core courses during the remainder of the UG degree courses. Enhanced Student Learning in Civil Engineering Courses Using CAS Calculators, Thomas Hu (CIVL) To develop teaching materials using a new solution methodology for engineering problems using a new class of calculators embedded with Computer Algebra Systems (CAS). Assessment Strategies in Problem-based Learning, Mike So (ISMT) To develop assessment strategies to align effectively teaching methods and learning outcomes to assessment in problem-base learning. A Learning-by-Example Learning Module for Programming Courses, James Kwok (ISMT) To develop an online learning module delivered through LMES to help business school students to learn their first programming language in a more effective way by means of learning-by-example. Biochemical Basis of Parkinson's Disease & the Therapy, Yi Fan Han (BICH) To develop a 3D animation software package for biochemistry and biology major students to demonstrate the biochemical basis of Parkinson's Disease including the symptoms, pathology and therapy. Virtual introduction to Biomolecules and their metabolic intermediates, Robert KO (BICH) To offer a step-by-step approach to understanding the structures, properties, and functions of major types of biological molecules. Adaptations An adaptation is any attempt to modify or enhance an existing innovation, which could be a teaching aid or method, implemented in any one of the previous CLI projects. Production of advanced illustrative animations of finite element simulation for teaching in structural mechanics and stress analysis courses, Ricky Lee (MECH) To generate more advanced illustrative animations of finite element simulation for teaching in structural mechanics and stress analysis courses.

Extending GONG to enable audio podcasting for teacher/learner communication, Jogesh Muppala (COMP) 5 . To adapt the GONG tool to enable bi-directional audio communication

between the student and the teacher with the content available as cast Adapting Gong for LANG208, John Milton (LANG) To produce a simplified audio recorder that can be incorporated into any web page. Bioinformatics in Bioengineering: Direct Experience with lnteractive Learning, Kathy Luo (CHEM) To redesign the software and website developed in the original project for students in bioengineering. Adaption of Podcasting techniques for a postgraduate Computer Science Course, David Rossiter (COMP) To provide students access to the lecture material via a podcast. If you are interested to find out more about the projects, please visit http://celt.ust.hk/cli/. Tech Camp 2006 Winter A new round of Tech Camp, aimed at enriching teaching About 40 teaching staff participated in the five mornings staff's information and communication technology (ICT) and one whole-day session, they learnt techniques in basic skills in a fun and exciting environment, was held again image creation and editing, video production, creating on 11-19 December 2006. With the theme, Click! interactive presentations, instructional websites Edpedition, participants were able to learn a wide and e-portfolio by using built-in tools including range of skills with technologies that support their MS Powerpoint, MS Frontpage, Adobe instruction. Photoshop and Adobe Premiere. 'Useful & interesting' 3 'It can really provide me more chances to practise.' r 'Opportunity to learn more advanced skills, leading to more professional web site design possibilities' 4 'The practice session was useful and tailored to our own preferences.' good workshop with friendly support from the instruction please visit htp://celt.ust.hk~instr/MiniTechCamp/home.html team.'

Recent Events Highlights TA Enhancement Program Spring 2007: Teaching Assistant Enhancement Program for new or Year 1 TAs at HKUST started on 25 January 2007. For details, please go to http://celt.ust.hWtaL Tech Camp 2006: Tech Camp was held from December 11-19, 2006. Learning experience incorporating image editing, video production, creating interactive presentations and building up instructional websites was brought to a group of participants. For more information, please refer to Page 12 of this edition. Seminar: Effects of Problem-based Teaching and Learning Activities on Student Learning: Conducted by Prof Mike So, ISMT, the seminar was well-received. Please refer to http://celt.ust.hWseminar/archive.html for more information.