د/ايمان زغلول قاسم

استاذ تكنولوجيا التعليم المشارك بكلية التربية بالزلفي

EVALUATION

EVALUATION
We have designed and performed user studies to evaluate
the effectiveness of our Lego-based approach on teaching
security protocols. The main evaluation goal was to compare
our Lego-based approach with the traditional text-based
approach from different educational aspects. The results of
these studies have provided important information to us
on the advantages of the new Lego-based approach, as
well as useful clues to improve this visual-based scheme.
Our evaluation plan consists of the following two
portions: an informal survey for gathering feedback on
the general Lego-based approach, and a formal user study
for assessing the specific performance of the Lego-based
and text-based approaches. The following first describes the
informal survey, which shows significant interests in using
Legos in class from students. Later, we present two
experiments in our user study and discuss their results.
6.1 Survey
Due to the limit of available time in our class, we designed a
brief survey to assess the general interest of students for
Lego-based approaches. Our hypothesis is that an interactive
tool based on a popular toy concept would pique the
interest of students in computing majors more so than
traditional text-based methods. We believe the positive
results indicate that visual approaches can better encourage
students to study challenging and abstract security theories.
The following lists the subjects, procedure, results and
discussions of our survey.
6.1.1 Subjects
Our subjects include 23 student volunteers from the
“Introduction to Information Security and Privacy” class
YU ET AL.: 3D DIGITAL LEGOS FOR TEACHING SECURITY PROTOCOLS 131
Fig. 12. Example results demonstrate that our approach can visualize various security protocols.
Fig. 13. The message, S ! A : fB;Ks AB;N A;N BgKs AS; fA;
Ks ABgKs BS, visualizes in different primitive styles. Our Lego
construction approach can switch among different primitive styles in
real time.
at UNC Charlotte. The majority of our subjects are juniors
with computing backgrounds. Since these students have
learned security protocols using the text-based approach
throughout the semester, they are all equipped with basic
knowledge of security protocols and are familiar with the
text-based approach.
6.1.2 Procedure
Before the survey, a 15 minute introduction of our Legobased
approach is given to the subjects. Since these
students are familiar with the text-based approach, we
concentrate on explaining how the designs of digital Legos
can be used to teach the primitive and protocol relationships
in general security protocols. We also demonstrate
the digital Lego system and its interaction methods. After
the introduction, we answer questions raised by the
students for about 10 minutes.
During the survey, each student is given a copy of the
survey questions and instructed to take as much time as
they need to finish. The survey is in the form of multiple
choice, Likert-scale, and free response questions. The
questions are used to assess the interest of students on
general Lego related issues. Fig. 14 shows the four multiple
choice questions, and Table 2 shows the four Likert-scale
questions (the six scales are strongly agree, agree, slightly
agree, slightly disagree, disagree, and strongly disagree).
6.1.3 Results and Discussions
The results of this survey indicate a strong motivation of
students to combine text and visual based approaches to
learn security protocols. Considering that the students
participating in this survey have only been introduced to
the Lego-based approach shortly, we think that they may
have questions and concerns on the details of digital Legos.
Even so, a majority of the students still choose the Lego-based
approach in addition to the traditional text-based approach.
As shown in Fig. 14, all the students consider themselves
to be visual learners, which shows a unanimous
interest in improving the traditional text-based approach.
About 91 percent students prefer to learn security protocols
as visual entities and imagine protocol scenarios in visual
forms. This number indicates a wide acceptance of visualbased
education tools. Also, in the last question “I feel I can
learn best by digital Lego, text, or both?,” none of the
students chose the text-based approach and about 65 percent
chose a combined Lego and text-based approach. Since
our system is able to show the plain text of protocols as
well as digital Legos, our system design matches the
interests of students.
Table 2 shows our Likert-scale questions and their
results. If we use the scores 0 to 5 to represent the choices
“Strongly Disagree” to “Strongly Agree,” the averages are
4.3, 3.35, 3.45, and 2.75 for questions (a)-(d), respectively.
For the first question, “I feel I can learn protocols by a visual
approach,” the average score 4.3 shows a strong confidence
in visual-based approaches. For the last question, “I feel I
can learn security protocols by a text-based approach,” the
average score 2.75 is just a little bit higher than neutral.
Since these students have been taught with the text-based

الوقت من ذهب

اذكر الله


المصحف الالكتروني