Last week, we presented and demonstrated Kritikos to delegates at the 5th International Materials Education Symposium, at the University of Cambridge. We were pleased by the positive reaction that we received and the number of enquiries as to if/when the full functionality of Kritikos would be available to students outside Liverpool.
This morning the team sat down to discuss possible ways forward and agreed in principle that we might be able to work with a small number (3-5) of partner institutions in trialing Kritikos, possibly on the basis of an individual course or module. By way of some guidelines:
- We are looking for a module/course with between 50 and 100 students. If appropriate for the host institution, we could consider a subset of students from larger classes.
- We would require a breakdown of the module/course into its main topics, so that students can relate resources to those topics (rather than the module as a whole)
- We’d aim to trial from Sept-Dec 2013
- At present we have only developed the user interface in English, though the actual search does work in all languages.
- We would need to cover the costs we incur from Google for use of their custom search engine. This is currently charged at the rate of $5 per 1000 requests. There might be some modest additional charges to cover our costs, but the trial would be conducted on a not-for-profit basis.
Please contact Andrew Green [amgreen at liv.ac.uk] if you are interested in participating in a trial. Please note that we will operate on a first come, first served basis.
Tim Bullough, Andrew Green and Adam Mannis will be presenting Kritikos to the 5th International Materials Education Symposium to be held in Cambridge, UK April 4-5, 2013.
Talks, workshops, and poster sessions will focus on four key themes:
- Engaging Student Interest
- Crossing Disciplines: Design & Architecture
- Emerging Opportunities: Sustainable Technology
- International Perspectives
The abstract for the presentation is here.
Today we had a very useful first meeting with Peter Reed from the Faculty of Health & Life Sciences to discuss the possibilities of extending Kritikos for use by students in the Medical School. We recognise that whilst much of the functionality of Kritikos is readily transferable beyond Engineering subjects, we will need to look carefully at how medical students would want to associate learning resources with distinct areas of study (module, topic, learning outcome, etc).
We also discussed how new features could be incorporated; for example how advanced search options might be used to limit results to certain domains, or how in due course, results could be limited to those with existing Learning Registry data attached to them.
We look forward to the possibility of close collaboration with Peter over the coming months and demonstrating the full potential of Kritikos.
We had a first team meeting today to discuss how we go about creating our evaluation video and to draft a storyboard for it. Luckily we have expretise within the team in getting good pictures and sound; and the draft script was created today.
We will draw on the technical guidance provided by JISC Digital Media (JDM), and use a 10-point list provided by them to ensure a level of consistency between all project videos from the JISC Content Programme.
This post presents the results from the online survey that the students completed after road-testing the initial iLIKE and ENGrich user interfaces last week. Some feedback from the ENGrich team is included too.
iLIKE – Module Tab
Is the Module tab potentially useful?
- Good news
Is there any missing or redundant functionality to the Module tab?
- I didn’t find the missing or redundant module tab
- When searching for some modules such as Aircraft Capstone Design, this did not appear whilst other modules did. This may be because the module title has changed but it could not be found even when searching for similar names.
- The results could be automatically linked to the upcoming lectures in that module, effectively a search of pre-defined terms set by the module co-ordinator linked to the module running order.
- Yes, we see the potential for this; the search results would need to be more ‘granular’ than at a module level. Our system would need to read weekly details from VITAL (Liverpool University’s installation of BlackBoard) to narrow the results even further.
- I feel that the iLIKE widget could be linked directly to the course material put on by the lecturer as well. There should be an easy button placed onto the front screen that people can use to add extra material that they think other students should find useful.
- This would require full integration with VITAL/BlackBoard, which isn’t feasible in the immediate future.
Is the display satisfactory; any suggestions?
- One potential problem I found was the number of tabs that could be opened during a 10 minute session, it did become quite tab heavy!
- Yes, we’ll aim to limit the number of windows/tabs that open.
- With only 3 results displayed in the preview results, users might be led to believe that these are specifically filtered i.e. most popular. If this isn’t the case, maybe it could be?
- In future, we aim to order thumbnails by most popular first for the Module tab, and most recent first for the My resources tab, when we implement it.
- I think that the layout is clear to understand and seems easy to use.
iLIKE – Search Tab
What does the Search tab do?
- have been sorted out, related to engineering, can be used for finding useful materials by students’ rating. A Google custom search engine based academic search tool
- The search tab allows a key word search to select a presentation/video which will potential display relevant material. Currently the search is either for presentations or videos. The system communicates with the Google search engine, and then displays 10 results per page. The results may have been recommended by another student which will be indicated with a green and blue arrow.
- It uses a Google custom search to narrow the search to visual resources and then overlays these results with any information in the learning registry.
- The search tab looks through information on Google that is relevant to the word that you have placed in.
How does this Search differ to Google, Bing, Baidu, etc?
- it is more academic. it has more limitations of materials.
- This search focusses on engineering topics, and allows users to interact with the search engine which will moderate the search results with the intent of over time, seperating the good from the bad.
- It only searches visual resources, but includes information about that search result from the learning registry – if the information exists.
- It should bring up information that is more relevant to engineering than the usual Google page and therefore should be more useful.
Is there any missing or redundant functionality you can think of?
- Possibly a different approach may be to pre define the module you are searching for, then any searches that you consider will be linked to this module, you may then have the option to rate the relevance of the document/video to the subject but if it applies to another module, make this option possible as well.
- Yes, we see the potential usefulness here. If search results are presented in the context an individual module, then you could tick checkboxes for all the relevant results and submit the ‘alignments’ in bulk, thereby saving you time.
- Pupils should be able tyo indicate which pages that have come up are the most relevant and which they think is still not relevant to engineering.
- Yes, as above, a system of checkboxes that can be ticked to indicate non-relevance might be efficient.
ENGrich Search/Results Page
What does this page do?
- This is the filtering process of the program and suggests relevant documents/videos and ranks from 1 to 10 on each results page.
- Displays all the results from the iLIKE search.
- This page displays all the results that have come up when you searched a topic.
Is there any missing or redundant functionality you can think of?
- An issue may be the ranking system adopted. A document may be very relevant for a year 4 student but too much detail for a year 1/2 student. Will this be taken into account as this may affect the quality of results that the user obtains. Also the number of tabs that can be opened can become a bit of a headache. I found I could easily close the main results page when I just wanted to close a powerpoint. Maybe a ‘challenge’ could be added to ensure they wish to close the main page.
- We think that this is exactly why we need your comments, which can be shared with the community.
- See comment above regarding tabs
- The results could be split into 2 sections side-by-side: one shows Google results which don’t have information from LR; the other has only results which have LR information.
- At present, the project is about integrating LR into Google search, thereby ‘engriching’ the results. At this stage, we don’t want to develop separate search engines or present their results separately
- This should then be vetted and if an independant person feels the same then there should be an option that stops this website from appearing when people search again.
- Indeed, our algorithm for ordering results will take into account the number of times a resource has been marked ‘not relevant’. Such resources won’t be displayed in subsequent searches.
Is the display, design and layout satisfactory
- The layout is good, text size is fine and the simple design makes it straight forward to navigate.
- Some way of clearly showing, maybe using the Google logo, that this is still a search of the whole Web through Google, not just of liv.ac.uk resources for example.
- Yes, we absolutely agree, thank you
- I feel that the display and layout is very easy to understand and use.
ENGrich Resource Detail Page
What does this page do?
- This gives a preview of a selected document and supples information on whether the document has been reviewed before.
- Displays information relevant to the search result selected.
Is there any missing or redundant functionality you can think of?
- If I find a useful presentation this time, I want to use it next time. I prefer having a link directly from my profile so that I could find it straightforward.
- Yes, the planned ‘My resources’ tab in iLIKE will do exactly that and link to a page in ENGrich which shows all your resources.
- The preview may be a little too small that it requires opening onto a full page. So it may be worthwhile to increase the size of this otherwise it does not offer much use. Would it be possible to save from this page directly instead of having to open and then save?
- For some formats (PPT, DOC, PDF and possibly images) we can provide a “Save as…” button. This won’t work for videos or Flash.
- Some way of the saving this search result, like bookmarking or adding to favourites, but built into the student portal/iLIKE channel/VITAL.
- By interacting with resources, they will automatically be added to ‘My resources’ in iLIKE. VITAL/BlackBoard is outside our control.
- If its possible to show who the source belongs to eg. a university or organisation? Instant recognition of likely reliability of result.
- We are adding the web icon for the source organisation, to help with ‘brand recognition’.
Is the type and format of information presented useful?
- the left side is the presentation or videoes and the right side is only a link to the original website. why don’t we make the space for the link smaller and make the space for the presentation or videoes bigger?
- Yes, we’re still working on the layout and will use the space more efficiently.
- As stated above, the size of the preview for some font sizes may be too small, which will make the preview useless.
- As the LR becomes more popular, a tabulated form of the “What others are saying…” section might be better. Could be information overload otherwise.
- Yes, absolutely. We’re discussing ways of aggregating data in a compact and clear way, e.g. “John Smith and 21 others recommend this resource for the Spaceflight module (AERO319).”
Are there any other ways you would like to “interact” with resources
- for example, I want to find something about Abaqus. It would be good if ilike is connected with the company who design this software. because they publish many things on their website. sometimes I only need this intermediary (bridge) for finding useful things.
- Use normal Google, etc. for finding company information. ENGrich is fine-tuned for finding and interacting with learning resources for Engineering
- Maybe a twitter account that ‘tweets’ whenever a useful document has been found. For example “@CiviliLikeLivUni just recieved a recommendation for CIVE430. by …” I don’t think facebook would really work. Like I said at the session I am more than happy to help as it will be improving student experience in the future. I look forward to the next session. (D Mount)
- Option to share via Facbook/Twitter or add to a resources section in VITAL/BlackBoard.
- We’ll add share buttons for Facebook, Twitter and LinkedIn. VITAL/BlackBoard is outside of our control.
Are you willing to come to future testing sessions?
Over the course of the summer, Engineering students from the University of Liverpool have been identifying visual learning resources relevant to their studies and have been entering data into our temporary database via a Microsoft Access (screenshot below).
Access form for students to enter details about online learning resources
We will then use these data to construct metadata and paradata statements to publish into our Learning Registry node.
With another week passing of creating the visual search engine our database reached the initial target of 25,000 links! This is a memorable day as it is also last day at work for one of our long term workers. We wish her a fantastic career in Civil Engineering as she will be starting her first job as a graduate student! Congratulations!
Even though, we`ve reached our initial target of 25,000 links we will keep working for another two weeks. Now our team will consist of only 3 people but we are hoping to cover all the key engineering modules` syllabus by the time our summer placement finishes. The hope and expectation is that the number of entries in the database will still grow beyond what our team has achieved thanks to all other engineering students that support our goals and are willing to help. But more about this later…
I hope you have been enjoying the Olympic spirit as much as we have! Our fact of the day is: Olympic rowing champions Tom James and Pete Reed (who won gold in the men’s coxless four) studied Engineering, as did George Nash (who won bronze in the men’s pair). Maybe we should get an office crew together!
Our numbers have been depleted this week, with a few students having finished their placements and others assigned to different tasks. Nonetheless, the 3 of us who are still ENGrich through and through are steadily progressing! We now have well over 22,000 resources logged having broken the 20,000 barrier at the start of the week. We remain on-track to register resources for every engineering module taught at Liverpool during our time on this project.
Towards the end we have been tackling more diverse and specialised modules such as biomaterials, spaceflight, smart materials, manufacturing, energy & power systems and laser processing. The inclusion of modules like these is just as important as the more common ones as they widen the potential reach of the finished ENGrich project, when students from other universities (globally) can use and add to the database. And now for the customary useful link! This interactive animation of an engine cylinder is very useful. It enables students to change engine settings and parameters and see how this affects performance. They could also attempt, for example, to maximise the power output or efficiency of the engine without causing knocking to occur in the engine, which is a serious problem). This is a great way to learn about the compromises required in internal combustion engine design
Over the past week we have been updating the module list, which has included the addition of diverse topics such as Biomedical Materials and Spaceflight. The team has been reduced down to only a few members,but we are still plodding on with linking. One aspect of the project that has really taken our notice is the global scale of our resource database. So far we have gathered links from India, Australia, China and the Americas, just to name a few! And whilst a majority of these links are from academic sources, many have also been made available by engineering organisations and individuals. We are now approaching 18,000 links, so well on our way to our target.
One of our team graduated with a Masters in Civil Engineering last week,
so in light of this we have provided a sky scraper game.
Will your building last?
Today the 5th week comes to an end with over 15,500 links recorded in the ENGrich database. It`s been a cheerful week since the boys went on holiday and we were working in all girls company. A lot of interesting subjects were covered which didn`t affect our efficiency at work. A lovely red helium balloon from the Active Learning Lab joined our party too and lightened those sad rainy days that were hunting England recently.
The ENGrich site now shows thumbnails of our links and we are getting closer to obtaining our goal in creating a visual search engine. We hope that as a new academic year commence in September the interface will be perfected and the total number of links will surpass 30,000.