Students want more and faster feedback on the texts they write. A research project in secondary school suggests that students can write better texts if they receive feedback from a computer program based on artificial intelligence.

Written by Ann-Kristin Johansen, Østfold University College

At Østfold University College (HiØ), researchers are working on developing an artificial intelligence program to help students write good texts, and the first prototype has already been tested on secondary school students at three different schools in Halden.

In collaboration with teachers at the pilot schools, the researchers developed a writing process for pupils on the theme of heroes. The pupils were asked to write a text about who they consider to be a hero and argue why. Based on the text they entered into the computer program, they received individual feedback in return. The actual writing process consisted of the students working on three drafts of the text and receiving feedback between each round, with the last draft being submitted to the teacher for a final assessment.

- It was designed as a writing process where students wrote their texts through multiple drafts and received feedback from the technology," explains Irina Engeness, professor and dean at the Faculty of Teacher Education and Languages at HiØ.

The testing of secondary school students is part of the project "Artificial intelligence (AI) for assessment for learning (AfL) to strengthen learning and teaching in the 21st century" (AI4AfL). The project has received NOK 12 million in funding from the Research Council of Norway and is run by HiØ in collaboration with Volda University College, Halden Municipality and Hypatia Learning AS.

The university, the municipality and Hypatia Learning are cluster partners in the Cluster for Applied AI.

Such clear differences

While some of the classes that participated in the writing process were allowed to use the new computer program in their text work, the other classes were to work without this help and instead give each other feedback along the way using the assignment criteria. The preliminary results indicate that the choice of method had a different impact on the final grades the students received.

- "At all three schools, the classes that participated in the writing process using the new technology have improved their grades from the first draft to the final version," says Engeness, who is leading the project.

In the classes that did not have access to the program, there were also changes in grades during the process, but not to the same extent. The researchers found that students who used the technology were more involved in the discussions about the feedback and how to further develop their texts in line with the feedback.

- "We saw that they used the feedback provided by the technology as a starting point for the conversations, and from a pedagogical perspective, the discussions in these classes were more content-rich. At the same time, we saw that the students in the other classes that didn't use the same technology were unsure of how to give each other feedback," says Engeness.

The quality of the feedback that students receive on schoolwork generally varies widely, according to the researchers.

- "Some students find that they only receive feedback such as 'write more', but they don't necessarily understand what else they should write," says Siv Gamlem, a professor at Volda University College.

Gamlem's research includes assessment and feedback and says it is important that students receive feedback as an integral part of the writing process. The preliminary findings from the project also confirm this.

- With this program, we want to give all students an equal opportunity to develop their work," she says and adds:

- Through this program, they will receive input on the content of the text and what they should write more about. This will help them to develop knowledge and understanding of their own writing process and to write their own texts.

Received significantly more feedback

After the writing process, the students were interviewed about what it was like to participate.

- "The students who used the computer program said that they really appreciated the feedback they got from it," says Engeness.

The students who did not get to use the new technology had a slightly different experience of the writing process.

- They stated that it was not easy to participate and found it difficult to find the right form of feedback to give and what to focus on. As a result, they became very cautious with their feedback," she explains.

According to the researchers, the project so far shows that the students who used the technology also received the most and most varied feedback. This finding is not unexpected.

- "It will not be possible to give all students this type of feedback integrated into a writing process if it is done by only one teacher," says Gamlem.

Through the technology, each student received as many as 30-40 pieces of feedback on each draft. According to the researchers, the teachers involved expressed concern prior to the workshop that the computer program would give students too much feedback. However, this was not the case.

- "When we interviewed the teachers afterwards, they were surprised at how carefully the students took in all the feedback. At the same time, they said that they would never be able to give the same amount to each student as they received through the program," says Engeness.

However, the researchers note that there are individual differences between students, and that care must be taken not to give too little or too much feedback. Students must be both able and willing to use the feedback they receive on their work.

Will work on improving the concept

The students who participated in the project stated that they liked the design of the program and found the quick response positive. They also expressed that it was positive to work in a process where they could improve their texts in several rounds.

However, the amount of feedback the program provides is one of the things the researchers will now continue to work on towards the completion of prototype 2 of the program.

- "We need to find a solution to how much feedback should be given so that it is motivating and not too much. We know that students who don't perform well will receive a lot of feedback that highlights mistakes, and this can be demotivating," says Gamlem.

She emphasizes that they must ensure that feedback is given in a type and amount that contributes to further development in the students' text work.

- "If they get too much, they won't be able to use it, which is why it's been important for us to interview the students participating in this experiment. We need to make sure that all students receive feedback that motivates them to develop and helps to improve their work," she says.

No new Chat GPT

The development of the computer program is in full swing, in parallel with the rather intense debate surrounding Chat GPT and similar services based on artificial intelligence. The researchers behind the multi-million euro project AI4AfL emphasize that they are not working on a separate Chat GPT version.

Except for the fact that the program they are now developing is based on AI technology, it has little in common with Chat GPT.

- These are completely different products. Chat GPT is a product that produces a text for the user, while in our case it is about taking an essay or text that the student writes and providing feedback on it. Only the AI technology behind these software products is similar.

This is according to Associate Professor Hasan Ogulat the Faculty of Information Technology, Engineering and Economics at HiØ.

According to Ogul, the first prototype of the program, which focuses on syntactic errors, makes little use of AI technology. The second prototype that will now be developed based on, among other things, the results of the testing done among school students, will make greater use of AI technology because it is about analyzing and understanding the actual meaning of the texts.

- It is important for us to emphasize that we are not developing another Chat GPT technology. The educational perspective is absolutely essential to the technology we are developing. It should offer precise and useful feedback on student submissions that can be addressed in the next draft. Such a writing process can be embraced by students as a good approach to understanding what it means to create a good essay," Engeness explains.

Strengthens the teacher as a guide

After testing the first prototype on secondary school students, the researchers have a comprehensive data set that will be analyzed and used to develop prototype 2. This will be tested on the same students during the fall in a workshop where they will work on both syntax and content. During the test project, the researchers collected film material showing the students at work and how teachers give feedback on their writing. They also collected a large number of essays in three versions from each participating student.

The data will be analyzed from both a pedagogical and a technological perspective. In prototype 2, the researchers hope to strengthen the program so that it can work even better for teachers from a guidance perspective.

- "We are all former teachers, and we know how difficult it is to collect data on students to find out what they are struggling with. This tool provides specific info about each student that the teacher can use to understand what the student actually needs," Gamlem points out.

At the same time, she emphasizes that the program cannot replace the role of the teacher.

- It's important to remember that there's one thing that students need that they don't get from technology, and that's emotional support. If you see students struggling, as a teacher you can show that you see them and offer both academic and emotional help and support. This program will give the teacher time and opportunity to use their skills and knowledge in a different way," she says.

Eli Haugerud from Smart Innovation Norway will lead the work to establish a new meeting place across the country's many clusters.

The initiator of the new cluster network is FIN - The Association of Innovation Companies in Norway, which has most of the clusters as members.

General Manager Trine Ellingsen says that many people have asked to meet to exchange experiences and discuss current issues. 

- The clusters are key to Norway achieving its export and sustainability goals. We want to create an arena for sharing and collaboration,' she says in a press release.

Project manager with a lot of experience

Project manager Eli Haugerud works as Head of Communities at Smart Innovation Norway. This means that she is responsible for the innovation company's three clusters, NCE Smart Energy Markets, Cluster for Applied AI and the DECOM cluster, as well as the smart city initiative.

Knowing the needs of clusters with different levels of maturity and in different areas is useful in facilitating the exchange of expertise and experience for very different clusters. 

- Sharing is important. The immature can learn from those with more experience, we can also bring in impulses from outside and facilitate new collaborations," says Haugerud. 

- Happy to give something back

 This year, one or two meeting places are planned, and work is now continuing on the design of the program.  

- We have a dialog with several clusters to understand their needs and what we should do together. Industry Week in Porsgrunn is an example of an arena where we are present, and we would like to hear from more clusters," says the new project manager.

Ellingsen of FIN has great faith in the new offer.

- "The clusters are important to FIN and we are happy that we can now give this back to our members," says Ellingsen.

Norway's leading experts in artificial intelligence have established a new competence center, and Smart Innovation Norway is a natural partner.

The new national competence center has been named Nemonoor. The aim is to help enable private and public sector organizations to adopt artificial intelligence (AI).

- Artificial intelligence is referred to as one of the most important technological developments of our time. To maintain a competitive business sector in the future, it is important to use artificial intelligence for the benefit of customers, says Kjell Reidar Mydske.

He is the CEO of Smart Innovation Norway, which has built up Norway's largest business cluster in artificial intelligence, the Cluster for Applied AI. This is an industry cluster for companies that use AI (artificial intelligence) in a business context.

Giving business access to the experts

Mydske is proud of the position that Smart Innovation Norway has built up in AI in a national and international context, which makes them a natural part of the Nemonoor initiative.

- Here, companies get access to the national team in artificial intelligence, he says, pointing out that it is easier for companies to test the potential of their business in AI when you have a competence center that is closely linked to the business community.

- "Because we have a broad European network and good knowledge of clusters, the public funding system and key players at home and abroad through our work in the Cluster for Applied AI, Smart Innovation Norway is responsible for the network and ecosystem in Nemonoor," explains Mydske.

The establishment of the new competence center is also a further reinforcement of the renowned Halden environment. One of the knowledge companies in the network is the Institute for Energy Technology (IFE), which is also one of the initiators of the new center.

Nemonoor is a service provider that will find funding and support businesses with very specific resources. The center will also connect actors at both national and European level who can benefit from each other.

- AI has never been more talked about, discussed or utilized than now. Therefore, it is increasingly important for companies, organizations and businesses to understand and exploit the potential of the technology, says Nemonoor coordinator Eirik Andreassen in Digital Norway, one of the other actors behind the competence center.

See photos from the launch at Rebel in Oslo (click on the images for a larger version):

• 

• 

• 

• 

• 

• 

• 

• 

• 

- The timing is good 

Machine learning models cut emissions on construction sites. Computer vision streamlines maintenance and improves the quality of Norwegian road fences. Image recognition technology gives fish farmers better tools to monitor activity in cages.

These are just some examples of applications of artificial intelligence in Norwegian business today.

- There has never been such a short path from research to application in any field. So much is happening, and it's exciting. Now everyone can use a little bit of artificial intelligence, even if they don't know how to program.

So says Inga Strümke. She is a researcher at the Norwegian Open AI Lab at NTNU and SimulaMet with artificial intelligence as her field of expertise.

- We are still at an early stage of development. No one knows the extent of how technology will affect society and the world we live in. It presents many opportunities, but also a number of challenges," she points out.

Strümke believes the timing for Nemonoor is good.

- In Norway, we have a high level of trust, a high degree of digital maturity and lots of data. There are many opportunities to learn from each other and build bridges. It is important to have openness and more collaboration across the board to do this in a good way," she says.

- AI could cut global emissions by a fifth

The consortium behind Nemonoor consists of some of the leading artificial intelligence communities in Norway;

NORA/UiO, Norway Health Tech, NTNU, SINTEF, Digital Norway, Smart Innovation Norway, Institutt for Energiteknikk (IFE) and Ålesund Kunnskapspark (ÅKP).

Morten Dalsmo is Executive Vice President of SINTEF Digital. He points out that there is a crying need in the business community for access to resources that deal with artificial intelligence.

- The business community has a lot to offer when it comes to sharing data and expertise across industries and sectors," he begins, adding:

- Artificial intelligence has the potential to reduce global emissions by a fifth by 2030. At SINTEF, we are committed to connecting industry to the research front and ensuring that artificial intelligence is adapted to multiple application areas.

Håkon Haugli, Director of Innovation Norway, is also clear that the potential for sharing in Norwegian business is huge.

- Nemonoor is a fantastic connection box between different businesses and industries. Then it's up to participants to go into it with the good questions and willingness to share. If they do, then this is a great initiative," he says.

Closer to European developments

Nemonoor is a European Digital Innovation Hub (EDIH, pronounced "eddi") and is partly funded through Norway's membership in the EU's seven-year DIGITAL program. This means that the services are provided free of charge or at a reduced price.

According to Andreassen, Europe is currently undergoing a major resource boost when it comes to artificial intelligence. The EU has currently committed to the creation of 136 EDIHs in all EU Member States.

Andreassen explains that Europe has the world's highest density of researchers, but the world's lowest proportion of start-ups.

- This is an indication that we are failing to transfer the knowledge that exists in research environments and clusters to industry," Andreassen says and adds:

- The technology should be made so accessible that everyone can use it. That's where we need to go.

Haugli also highlights the importance of being connected to the European arena:

- It provides enormous added value if we are able to make use of this position. That we manage to position Norway internationally as an exciting country when it comes to the use of artificial intelligence.

Strümke is clear that understanding and knowledge of what artificial intelligence actually is and how it can be used will be particularly important if Norwegian businesses are to be good at using the technology.

- I also wish we had more political leadership in this field, but now there is actually self-coordination. It's an attempt to solve problems together - and that's fantastic," she concludes.

FACT BOOK:

Nemonoor is a brand new national center that will build the competence and capacity of private and public enterprises to use artificial intelligence.

Nemonoor consists of eight partners from research, academia, innovation, networks and test centers:

• Digital Norway
• NORA - Norwegian Artificial Intelligence Research Consortium/UiO
• Norwegian University of Science and Technology (NTNU)
• Norway Health Tech
• SINTEF
• Institute for Energy Technology (IFE)
• Ålesund Kunnskapspark (ÅKP)
• Smart Innovation Norway

Due to its broad European network and its good knowledge of cluster operations and public funding, Smart Innovation Norway's business cluster Cluster for Applied AI will be a key contributor to Nemonoor.

The center offers its services to businesses in the fields of:

• Production
• Health
• Communities
• Energy

Nemonoor is a European Digital Innovation Hub (EDIH) and is funded by the EU.

European Digital Innovation Hubs (pronounced "eddi") are a key instrument of the EU's new seven-year DIGITAL program.

Norway has been granted two EDIHs, Nemonoor and Oceanapolis.

As a skills broker, Hermund Arntzen Dale has first-hand knowledge of how research can help companies solve their challenges.

Competence Broker is a free tool for developing business in Viken through research-related projects.

For several years, Smart Innovation Norway has represented one of seven brokers in Viken county. Hermund Arntzen Dale has recently taken on the role and is already well underway with both outreach activities and guidance of companies.

- Well suited as a competence broker

The aim is for business to develop through research-based innovation by helping business actors to identify areas where research can contribute to further growth.

Dale gets high praise as a skills broker from Violette Yi Qin, project coordinator for Skills Broker in Viken.

- We believe Hermund is a good fit for the role due to his work experience, educational background and personality. He has worked in various industries within energy and technology and has a good understanding of the public funding system both through academia and work in industry. The fact that he has work experience as an entrepreneur, we think is very positive - then you can forge closer ties with an important part of the companies, which is the main goal of competence brokerage. In addition, he is friendly and motivated, which we think is a success factor for being a good competence broker in Viken," she says.

The key tasks of a competence broker are to help companies identify problems that can be solved with research, find relevant partners and the right expertise in the research community, and find funding schemes that can co-finance the project.

- My background means that I have good knowledge of the public funding system, and that I know the needs and challenges of early-stage companies. I also find it very fun to tackle challenges in the business community and look at the opportunities that exist to develop new solutions while working with my areas of interest, which are technology, climate and the environment," says Dale.

- To become more profitable than they are today

The competence brokerage scheme is funded by the Viken County Council's Forregion programme, which aims to mobilize increased use of research as an innovation tool.

Based on the Research Council's three priority areas of sustainable transition and value creation, commercialization from research, and regional development, competence brokers have been established within urban, urban and regional development, health and care, climate, energy and environment, and technology.

Dale offers his expertise and advice in technology together with Per Olve Tobiassen from SpinOn AS in Kongsberg.

- The service is initially aimed at enterprises with little or no research experience. The premise is that development should take place through research. Together we will identify the problem and potential of the business, with the aim of creating returns and new jobs. We will simply find solutions that are more profitable than they are today," Dale explains.

Skills brokers exist in every county and the community works across municipal and county boundaries.

- I can easily connect them with the right competence broker in the right field and geographical location. The others do the same for me. In this way, we have a nationwide reach and, in practice, no limitations," says Dale.

Need for guidance

Most research and development (R&D) projects rely on funding schemes to be realized. And it is start-up companies that often have the opportunity to try out new ideas in response to changes in the market.

Unfortunately, figures from the Research Council show that small and early-stage companies are the most likely to have their applications rejected.

- Young companies naturally do not have experience in writing project applications, and many do not spend enough time on formulating themselves correctly or responding to what the call for proposals actually asks for. It can therefore be useful to get guidance from a competence broker when writing applications," says Dale.

The offer aims to give companies experience in R&D. The FORREGION Business Project support scheme is therefore central. Under this scheme, companies can receive up to NOK 200,000 for their research project.

Competence brokers also provide guidance on other funding schemes at the Research Council of Norway, the Regional Research Fund and other actors. However, competence brokers do not provide guidance on writing applications to Innovation Norway (IN)'s funding schemes because they have their own supervisors.

- We work closely with IN and can, however, refer to relevant personnel who can be of good help to the company, Dale explains.

Short route to skills and networks

The "competence broker" function was awarded to Smart Innovation Norway since the company already works with business, development, innovation, applied research, technology and clusters.

Read more about Smart Innovation Norway's two other business clusters:

• NCE SMart Energy Markets
• Cluster for Decommissioning and Repurposing

Dale has access to Smart Innovation's entire network and all the knowledge of its employees. This puts him in a particularly good position to assist Viken's business community.

- The way we work on a daily basis at Smart Innovation is not very different from the typical day-to-day work of a competence broker. We share and use methods such as facilitating workshops, uncovering needs, mapping competencies and identifying potential, so I have a short path to competencies other than my own," he explains.

Experience so far shows that the free skills brokerage offer has great potential to become known among business stakeholders.

A large part of Dale's job will therefore be to conduct outreach activities both directly to innovation environments, companies and individuals, and by being present at trade fairs, conferences and other meeting places where these actors participate.

- An important task will be to motivate innovation and development. Not everyone sees their own potential, but with my knowledge and network, I see opportunities where others do not. Nevertheless, it is crucial that the will to innovate comes from within, and another important part of my job will be to be realistic and make the right recommendations for further development," he says.

Practical information

There is no application process for using a competence broker, nor is there any cost associated with the service.

Interested parties can contact Hermund Arntzen Dale by e-mail or phone +47 909 68 284.

Artificial intelligence is becoming an increasingly important driver of productivity and efficiency in organisations around the world. It can help businesses automate routine tasks, predict future trends and help make better decisions.

Smart Innovation Norway, together with the Circular Packaging Cluster, has launched the MOMAI project to help companies in the packaging value chain adopt data analytics and artificial intelligence.

Through the project, companies can gain general skills and understanding of the possibilities for using AI, identify challenges and opportunities in their businesses. They can also be introduced to technology companies that develop solutions for each company's needs.

AI and data: The key to industry's challenges?

The European Commission has presented a comprehensive proposal for a new Packaging Regulation with requirements and provisions on waste, reuse and recycling. The regulation aims to reduce packaging waste, increase the quality and use of recycled raw materials in packaging, and set requirements for increased use of reuse solutions.

But how can this be done while satisfying product quality needs and customer requirements for functionality? Some of the solutions are likely to lie in data analytics and automated intelligent systems.

Data management is the first step towards circular development

Artificial intelligence can, among other things, make material use more efficient by ensuring that good decisions are made for a more circular flow of resources, help customise packaging and optimise logistics and distribution.

But building such systems depends on having access to good quality data.

Looking for forward-looking businesses

The Circular Pacaging Cluster and Smart Innovation Norway believe that companies in the packaging industry can gain significant business benefits from adopting data analytics and artificial intelligence. We are now looking for forward-looking companies that want to identify their potential to become more data-driven.

For more information and to fill in the contact form, visit the project website!

MOMAI (Mobilisation and Maturation for AI) is partly funded by Viken County Council.