Interviews with involved teachers and students
1. Aims
The adapted teaching and learning arrangements were tested in each of the participating countries pilot schools in the context of work package nr. 9.
This field test was accompanied by subsequent interviews with the involved teachers who were asked about the practicability of the teaching materials and about any observed learning success. Furthermore, an analysis of the involved pupils was conducted measuring the acceptance, the motivation and the self-evaluation of their learning success.
The implementation of the teaching and learning arrangements in the pilot schools and the succeeding evaluation took 5 months (November 2012 – March 2013).
2. Preparation
The first step during the preparation period was the development of the instruments for evaluation on a supranational level.
The analysis and capturing of lesson events are a very complex issue of this study and scientists have been discussing this topic for a fairly long time. The controversy is whether a “quantitative” or a “qualitative” data analysis method is most suitable for such a study or if a model which focusses more on teaching and learning should be preferred (cf. Van Buer, 1984).
STACHOWIAK (1980) established his thesis that no matter which analysis method is chosen it is impossible to evaluate all “original attributes” but rather only those attributes relevant to the interviewee can be evaluated (cf. Stachowiak 1980).
Consequently, the initial research intention has to be detected first to align the analysis method according to the intentions.
In this case teachers and participating pupils are supposed to be questioned and therefore both parties will be placed in the center of the model. Due to the different number of cases, the unlike questioning aims and epistemological interests the decision was made to use different instruments for the evaluation.
The teacher’s positions were gathered with partially standardized interviews. This method has two advantages: On the one hand it is possible to ask spontaneous questions without hindering the natural narrative flow. On the other hand, teachers could elucidate their own opinions and observations on difficulties, successes and improvement proposals regarding the teaching material without any limitations.
The partially standardized interview guideline was designed based on the listed aims in the project proposal and it was developed by all project partners at a conference in Bergamo. The interview guideline consists of three parts: After a brief reception with some background information some general questions about the importance of self and social competencies in the retail sector are asked. The following two parts refer directly to the individual teaching and learning arrangements. Here the focus is placed on aspects of practicability, specific difficulties, improvement potentials as well as on identifiable learning successes of pupils.
In addition to the teacher’s interview guidelines an evaluation instrument for the second and by far bigger target group, the pupils, had to be developed. In this case a paper-and-pencil interview with a standardized questionnaire seemed most suitable. The pupils were uninformed about the research interests of the EU innovation transfer project and therefore a critical qualitative reflection would have caused great difficulties.
The main focus is placed on the motivation of the participating students in a time series analysis in the context of the implementation of the teaching and learning arrangements (Category 1). Furthermore, the acceptance of the participants had to be analysed in regard to the respective self and social competencies. Therefor the relevant items of category 2 are features like length, comprehensibility and accessibility.
Finally, two questions are asked to inquire the self-evaluation of performance growth (Category 3). The basis of this self-evaluation is a formative evaluation which accompanies the learning process. This enables each participating pupil to conclude a so-called pre-post comparison (cf. Siemon, 2003).
The finalized pupil questionnaire has a feasible size with 14 to 15 questions per teaching and learning arrangement. The questions were deliberately asked in the language of the interviewees and unknown or foreign words were avoided. A four-level Likert scale was chosen to measure the attitudes. The scale offered numerous answer possibilities with a very positive (“???”) and a very negative extreme (“???”). A neutral position on the scale was left out on purpose because it has been verified that many test persons use these positions as a comfortable way to quickly end the questionnaire (cf. Greving, 2009).
3. Implementation
The implementation period was initiated by a personal visit of the project partners in the participating pilot schools in each country. During this occasion the adapted teaching and learning arrangements as well as the pupil questionnaires were handed over to the respective teachers and eventual questions could be answered. This personal meeting prevents possible fears and worries of the teachers and at the same time it enhances the teachers’ willingness to implement the arrangements expediently in their lessons.
Each teaching and learning arrangement should be applied in the field at least once per pilot school. None of the project partners should be present during the actual implementation. There are several reasons for this. The presence of project partners during the actual lessons would tempt the teachers to ask questions while using the teaching and learning arrangement for the first time. This would deform the research process. Furthermore, neither the teachers nor the participating pupils should feel like there are being controlled and observed.
In the course of the first meeting a date was set on which the standardized pupil questionnaires were handed over and more important the individual interviews with the participating teachers would take place.
During the implementation period, between November 2012 and February 2013, the project partners were always available to answer questions and solve obscurities.
4. Preparation/Editing and Evaluation
During the last phase of the research interview contents were analysed and in view of the upcoming optimization (cf. work packages number 10) the data was edited.
A common guideline for this optimizing process was agreed on during the conference in Bergamo. That way a consistent format (“Teacher Evaluation Sheet”) was designed in which each partner could note the most significant points of criticism and improvement proposals per teaching and learning arrangement made by the participating teachers.
The project coordinator united those results in a way that the country of origin was untraceable. By these mean redundancies were eliminated and the supranational character of the research was preserved.
Partial transcriptions of the recorded interview into the respective national language were made in advance. The most important contents were written down and thus could be used for the subsequent discussion.
Before the data evaluation of the pupil questioning took place, the collected data was codified and edited.
5. Results
In Germany three pilot schools were involved in the implementation of the teaching and learning arrangements. In all schools the teaching material was used multiple times. The teaching and learning arrangement “Dealing with Criticism” was tested six times, the teaching and learning arrangements “Team Competency” and “Time Management” eight times and “Self-Evaluation of the own strengths and weaknesses” five times.
Therefore, the materials were use twice as much as originally planned which increases the validity of the results considerably.
After the implementation of the teaching and learning arrangements all participating 22 teachers were personally interviewed. 433 pupils completed the standardized questionnaire (71 questionnaires for arrangement “Dealing with Criticism”, 158 questionnaires for arrangement “Time Management”, 101 questionnaires for the arrangement “Team Competency” and 103 questionnaires for the arrangement “Self-Evaluation of the own strengths and weaknesses”. This is also a sufficient number of samples to generate profound and significant findings.
5.1 Teaching and learning arrangement “Time Management”
The attitudes/preferences and improvement suggestions of the various teachers vary greatly between the four modules. The teaching and learning arrangement “Time Management” was the most criticized. The criticism is that the level for the new target group, pupils of vocational schools in the retail sector, is still too low and that the presented examples are too unrealistic. According to the teacher’s opinions pupils of vocational schools in the retail sector do not experience time pressure in their jobs. Instead of creating a schedule for only one day almost all teachers recommend to let the pupils plan an entire week instead of only one day. In this scenario there would be a lot more appointments and therefor the task would be more complex and conflict-ridden.
Another point of criticism relates to the teaching material which according to the teachers contains a lot of repetitions which quickly bored the pupils. The complexity of problems should be increased and the contents should be designed more diverse.
The formal structure of the teaching and learning arrangement was rated very positively. The access to the topic through Mark’s daily routine and the following transfer by creating a personal daily schedule was valued as expedient by the teachers. Helpful examples were named to increase the complexity of the outlined daily routine. They will be included in the optimizing phase.
Besides those learner-focused aspects there were also improvements suggestions made regarding some teacher-centered aspects. Teacher predominantly emphasized that hints for them should be further specified.
The pupils’ statements confirm their teacher’s statements. The adolescents rated their own motivation towards the teaching and learning arrangement as well as their willingness to create a personal daily schedule as very poor. In addition they criticized the relevance of the taught content for their daily (work) life. Mark’s daily routine was too unrealistic which causes a harsh decline in their interest.
5.2. Teaching and Learning arrangement “Self-evaluation of the own strengths and weaknesses”
The critic towards this teaching and learning arrangement was distinctly more positive in comparison to the former arrangement. The teaching and learning arrangement “Self-evaluation of the own strengths and weaknesses” was consistently rated as very positive and meaningful. The discussion of personal strengths and weakness together with the whole class was quite successful in some classes. In other classes the personal inhibition level was too high so that teachers led a higher number of personal conversations with individual pupils. This proves that the optional choice of one of the methods is very feasible.
No teacher used the certificate “Employee of the month” because singling out only one pupil could generate envy among the classmates. Consequently teachers plead to create a more general certificate and completely leave out the entry of a specific person.
The teachers’ positive attitude towards this teaching and learning arrangement was confirmed by the pupils. They perceived it as very important to know the personal strengths and weaknesses (cf. Figure 4). The acceptance of the contents was very positively ranked in all aspects (cf. Figure 5). Moreover they were able to detect a personal learning success (cf. Figure 6) which confirms the importance of this teaching and learning arrangement.
5.3. Teaching and Learning arrangement “Dealing with Criticism”
The teaching and learning arrangement “Dealing with criticism” was also judged rather critically. From the teachers’ point of view it should be noticeably compressed because it proved itself as too extensive and therefore impracticable. The included worksheets have an insufficient number of work orders. The content of the role play should be broadened without increasing the total capacity of the teaching and learning arrangement.
Contrary to this negative criticism the included photo story was appreciated by the teachers as a welcome alternation. The fact that the story takes place in a clothing store does not prevent the use of the story in retail classes of other branches.
The pupils’ acceptance of this arrangement was rated quite positively. But opposite to that the self- evaluation of their own learning success as well as their motivation towards this arrangement was rated decisively poorer. An explanation for this could be that some students were already familiar with the content of the communication models from their former German classes. To prevent these superintendents of the educational programmes should organize the contents in advance.
5.4. Teaching and Learning arrangement "Team Competency"
Finally the Teaching and Learning arrangement Team Competency was discussed. In this arrangement teachers had the choice between option A (cardboard box constructions) and option B (case study). Teachers who chose option A wanted to try something different and practical but had to find out that the construction of the cardboard box did not work with the given instructions. To improve this arrangement the instructions should be way more precise and a standardized DIN paper size should be used.
Contrary to that the case study was rated clearly more positive. The only criticism was that the case study could provoke more problems and conflicts to aggravate the team building.
6. Prospects
The qualitative questioning of the teachers and the quantitative evaluation of the participating students provided important and expedient results. Those insights from the field tests now will be used to improve the materials on the national and supranational level.
These results will be the key product of the innovation transfer project ReSeCo and therefor have to meet the highest quality standards.