The Internet and Real-Time Evaluation

Liz Lawley in Many2Many discusses the merits of blogging conference presentations, and describes the different types of presentations, from good speakers & good content to the reverse. The privacy of IRC or other media encourages criticism, and some critical thinking, as well as plain old heckling. I see this as a pretty good method to evaluate conference presenters, either as formative evaluation for improvement or summative evaluation, to ensure that they don’t get invited back if they can’t cut it. Blogging and chat seem to be better evaluation tools than "smiley" sheets that few attendees complete …

This reminds me of Conor Vibert’s competitive intelligence class at Acadia University, where he has students giving presentations on a business, while others are going online to question their claims, and other students are using chat to discuss the points without interrupting the speakers. It’s exciting to watch Conor’s classes in action at the Acadia Real Time Case Competition.

Edinburgh Scenarios on eLearning in 2014

I’m posting this information on the Edinburgh scenarios, because I believe it is important, and because I cannot find any more information on the results of the scenario building that took place in Edinburgh last month. Does anyone have any more information? I would like to keep the conversation going.

The eLearninternational 2004 World Summit recently wrapped up in Edinburgh. Prior to the summit, four scenarios were developed by a group of experts, including Jay Cross. These represent a good synthesis of the spectrum of possibilities for 2014. The four scenarios are based on two critical uncertainties, as determined by the working group:

1. Whether the role of technology in society empowers or frustrates; and
2. Whether power, influence and new ideas are primarily from established sources or from new and emergent sources.

The resulting scenarios from the quadrants formed by the intersection of these two axes are:

Web of Confidence: Technology advances, power shifts to emergent players
U Choose: Technology frustrates, power shifts to emergent players
Virtually Vanilla: Technology advances, power retained by established players
Back to the Future: Technology frustrates, power retained by established players

Each scenario describes the main attributes of Learners, Education, Corporations and Government concerning elearning in the future. Both the Web of Confidence and U Choose offer a potentially optimistic future for our own learning industry in New Brunswick, while the other two scenarios are less positive for a small region like ours. In analyzing these scenarios, one can find something plausible in each one. Each scenario will have its proponents and its detractors. There are a number of comments on each scenario online that can be viewed.

edinburgh scenariosThese scenarios offer some common ground for discussion, and we can ask the question: Towards which scenario should we strive? I would suggest that it is the Web of Confidence, with new technologies being used by nimble niche players to foster innovation and learning. On the corporate side, the Web of Confidence would see more flexible, innovative companies offering new forms of learning, while larger corporations struggle to match the ideas produced by start-ups and networked, self-governing groups. If we want the Web of Confidence, with empowered learners and decentralized education, to become reality we should do everything we can to support this vision for 2014.

Another way to look at the possible future is by using Marshall McLuhan’s Laws of Media (McLuhan, M. & McLuhan, E. ,1992, Laws of Media: The New Science. University of Toronto Press.) tetrads on elearning. McLuhan’s laws of media can act as a lens to analyze any medium. The laws of media present a way of analyzing and predicting the effects of a new medium such as elearning.

The laws are presented as a series of questions or probes:

Enhancement. What does the medium enhance, extend, enlarge or intensify?
Obsolescence. What does it make obsolete? When an old medium enters its obsolescence phase, it becomes more ubiquitous, often changing from a utilitarian to a recreational role (e.g. fountain pens).
Reversal. When something is extended beyond the limits of its potential, its characteristics are often reversed. For example, cars which promote greater freedom, when multiplied to the extreme can result in gridlock.
Retrieval. What medium that was previously rendered obsolete does the old medium retrieve from the past? This is usually something from the distant past.

Using these laws we can examine the four components of the Edinburgh scenarios (Learners, Education, Corporations, Government). According to Federman & deKerckhove “the most interesting and revealing exploration comes from the RETRIEVES quadrant“. This is where the money is, in developing potential scenarios.

elearning (Learners)
1.Enhances independence
2.Obsolesces sitting in a classroom
3.Reverses into information overload, isolation
4.Retrieves diaries (web logs)

elearning (Education)
1.Enhances its reach.
2.Obsolesces teacher as “sage on the stage”
3.Reverses into commodification of learning
4.Retrieves community involvement (parents involved via elearning from home or office)

elearning (Corporations)
1.Enhances just-in-time training
2.Obsolesces just-in-case training
3.Reverses into over-prescription and overload
4.Retrieves sense of community (virtual communities of practice)

elearning (Government)
1.Enhances learning for all
2.Obsolesces physical infrastructure
3.Reverses into “one size fits all”
4.Retrieves the “town hall” (individual access to government)

This is only one exercise, and is limited in its insights, as the tetrads should be used, re-used, refined, questioned, and further refined before drawing any conclusions. McLuhan would always recommend deeper probing. But what can this initial probe tell us? First, we could say that individual freedoms are retrieved by elearning. This shows that one of the strengths of elearning is in fostering individual freedom to learn. One result of this retrieval could be more fragmented markets, composed of many individuals. In turn, this could create more room for multiple niche offerings. This form of retrieval supports the case for the Web of Confidence scenario.

At the same time, there is the potential for the reversal effects of elearning shown above. The reversal effects could result in the bleaker scenarios of Virtually Vanilla or Back to the Future. Much of this future is ours to shape, but by seeing the potential effects of our decisions today, we can make the right decisions for the long term.

Better than Bloom?

A recent Google search for “Bloom’s taxonomy” reveals over 50,000 hits.

After almost 50 years, Bloom’s taxonomy is still being used by educators and trainers as a pedagogical tool for the analysis of learning objectives. Originally designed as a method for the development of test questions, the six levels of the cognitive domain (knowledge, comprehension, application, analysis, synthesis, and evaluation) have become almost standard in the “learning business”.

I used Bloom’s taxonomy about ten years ago, while developing an estimate for the cost of CBT development.

We assumed that the higher the level, the higher would be the cost. With hundreds of performance objectives, we quickly reduced the six levels to three, but I now realise that there could have been many other ways to address the problem.

For instance, in Problems With Bloom’s Taxonomy (PDF) Brenda Sugrue states that Bloom’s taxonomy is invalid, unreliable and impractical. According to Sugrue, the six levels of Bloom’s taxonomy for the cognitive domain ” … are not supported by any research on learning.” Basically the taxonomy was a “best guess” by some knowledgeable educators of the time. The six levels make for nice matrices and provide a simple tool for analysis and evaluation, but Sugrue shows an even more effective way to create a Content-Performance matrix. Sugrue is not the only person who considers Bloom’s taxonomy passé. Another critic of the taxonomy is Robert Lewis [dead link] Professor of Knowledge Technology at Lancaster University.

Unfortunately, old chestnuts like Bloom’s taxonomy stay around longer than they should, because after a while we take them for granted. Every once in a while, it’s good to take a long, hard look at our practices, and make sure that we are using proven methods, and not second-rate tools.

Update: Donald Clark adds to the fire, stating that, “The weaknesses are so strong that it is simply sensible to abandon it altogether.”

Also, Carl Bereiter & Marlene Scardamalia of OISE offer Beyond Bloom’s Taxonomy: Rethinking Knowledge for the Knowledge Age.

Quote of the Day

Here’s a gem from Jay Cross:

"This morning, when separating the e-wheat from the e-chaff in my mailbox, something compelled me to click open a mass mailing from ASTD." [bold is mine]

How much of our daily routine is spent doing this?

Books I Would Like to Read

This is my constantly changing list of books that I’m thinking about reading. Comments and suggestions are very welcome. While my list is relatively short, you can visit the GBN Bookclub Selection for a much more comprehensive list.

Inventing the Organizations of the 21st Century, Edited by Thomas W. Malone, Robert Laubacher and Michael S. Scott Morton, (2003)ISBN 0-262-13431-4

The Future of Knowledge: Increasing Prosperity through Value Networks by Verna Allee (2002) Butterworth-Heinemann (ISBN: 0750675918)

The Clock of the Long Now: Time and Responsibility: The Ideas Behind the World’s Slowest Computer by Stewart Brand, 2000 (ISBN: 0465007805)

Breakaway: Deliver Value to Your Customers–Fast! (2002) by Charles L. Fred (ISBN: 0787961647).

Survival Is Not Enough: Why Smart Companies Abandon Worry and Embrace Change by Seth Godin (2002) (ISBN: 0743233387).

The Rise of the Creative Class: And How It’s Transforming Work, Leisure, Community, and Everyday Life (2004) Richard Florida (ISBN: 0465024777) Perseus Books.

OpenOffice.org for Dummies (2003) (ISBN: 0764542222)

Maps of Time: An Introduction to Big History, David Christian, 2004 (ISBN 0-520-23500-2) University of California Press.

The Ultimate Competitive Advantage: Secrets of Continually Developing a More Profitable Business Model by Donald Mitchell, Carol Coles, Publisher: Berrett-Koehler Pub; (March 2003) (ISBN: 1576751678).

Surfing the Edge of Chaos: The Laws of Nature and the New Laws of Business
by Richard Pascale (2001) Three Rivers Press , (ISBN: 0609808834)

Strategic Planning for Nonprofit Organizations: A Practical Guide and Workbook by Michael Allison, Jude Kaye, 1997, John Wiley & Sons Canada; (ISBN: 0471178322).

Strategic Tools for Social Entrepreneurs: Enhancing the Performance of Your Enterprising Nonprofit by J. Gregory Dees, Jed Emerson, Peter Economy(ISBN: 0471150681).

Nonfiction Matters: Reading, Writing, and Research in Grades 3-8 by Stephanie Harvey Publisher: Stenhouse Pub; (1998) (ISBN: 1571100725).

The Innovator’s Solution: Creating and Sustaining Successful Growth by Clayton Christensen, Michael Raynor (2003) Harvard Business School Press; (ISBN: 1578518520)

Leonardo’s Laptop: Human Needs and the New Computing Technologies; by Ben Shniederman; (ISBN: 0262692996) 2003; MIT Press.

Persuasive Technology: Using Computers to Change What We Think and Do; by B. J. Fogg; (2002) Morgan Kaufmann; (ISBN: 1558606432)

Performance Intervention Maps: 36 Strategies for Solving Your Organization’s Problems by Sanders & Thiagarajan (ASTD 2000) (ISBN 1-562862-93-6)

Handbook of Human Performance Technology: Second Edition
Improving Individual and Organizational Performance Worldwide
Edited by Harold D. Stolovitch & Erica J. Keeps (ISBN 0-787911-02-9); ISPI/Jossey-Bass/Pfeiffer/1999

Visual display of quantitative information, E. Tufte, (2001)

Leveraging Communities of Practice for Strategic Advantage: Saint-Onge & Wallace (2002). Boston: Butterworth-Heinemann.

Laptop Schools

I have generally been against the use of technology for technology’s sake, and this includes laptops in schools. An article in Syllabus has raised a good point to make me question my anti-laptop stance. According to the author, having laptops available to all students provides more opportunities for advanced students. "As the schools embrace full access to online resources, they are importing services and resources. They are also giving kids access to online Advance Placement (AP) courses that are produced and distributed by colleges and corporations. These school districts could never afford to support as many AP students as is possible electronically." Perhaps I’ve been wrong.

Systemic Causes of Societal Problems

Rob Paterson, on PEI, writes about the systemic problems facing island society, in a letter to the Provincial government. “When a business is in real trouble, it is because it has a systemic problem. Trimming here and there will not save it. So it is with us. Prince Edward Island has a structural problem related not to its physical assets but to its human capital.”

This letter is worth the read, as Rob’s view is that the root causes of complex problems cannot be addressed by focusing on the symptoms. This is also the view of human performance technologists. Rob’s points are pertinent to one of my projects, where we are making recommendations on how to grow the New Brunswick learning industry. Our key for sustainable development is to find out what is at the root of the problem/challenge. For PEI, Rob says that many children are already challenged before they enter school, because learning and literacy skills are developed prior to age four. All of the interventions to develop literacy and learning skills after this age are treating the symptoms, not the root cause of the problem.

This is the same kind of root cause that I am looking for. What are the core components of an industry that will ensure sustainable growth? Is it infrastructure, government policy, access to capital, R&D, marketing & branding, an educated workforce, etc.? My suspicion is that the basics are the most important part. For instance, I moved to Sackville because there is a university, a local hospital, a good elementary school and cheap housing. I have stayed here because of high-speed internet access and two relatively close airports.

Are there basics that need to be addressed in order to grow an industry? In the case of PEI, Rob makes the point that early childhood interventions are critical for a sustainable society. Are there other pieces of the puzzle critical to grow a small industry, or should we focus on the region or province instead, and let industries grow in what we have cultivated? This is one of my current challenges, and I would appreciate any feedback.

Social Networks & Corporate Learning

Jay Cross of the newly renamed Emergent Learning Forum, discusses social networks and their value for corporate learning. This is the frontier of the Internet, building on the writings of Malcolm Gladwell (author of The Tipping Point)and others, on how loose social ties are more important than strong ones for learning, as they introduce ideas and people outside of our usual social circle. Social networks, communities of practice, expertise locators, etc. have more potential and utility in this medium than centralized systems such as LCMS. If you have an interest in this field, then join the Forum, it’s free.

Learning, Work, and Technology

This is where I post my thoughts and comments on ideas, events or other writings that are of a professional interest to me. Current areas of interest include social networking applications, like blogs, wikis and the use of RSS feeds, which is one reason why I have this blog: to practise what I preach. I’m also interested in the use of open source software platforms for learning. The development and nurturing of communities of practice online is another area of applied research that interests me.

My previous blog is still available as an archive.