ai, ml & data engineering, data science, neural networks, deep learning, artificial intelligence, machine learning, computer vision

A new dataset called the CLEAR Benchmark: Continual LEArning on Real-World Imagery aims to establish a consistent image classification benchmark for future research in continual learning.

Retraining deep networks to adopt new data or tasks (e.g. new labels) is of practical importance in machine learning engineering. The challenge may establish itself distinctly in different fields. For example in time-series forecasting, network training has to be carried out periodically due to the observed data drift, hence one may desire that the previous training iterations have a long-lasting positive effect on the test-time performance as the new ones are being scheduled. In computer vision, it may be necessary to add new labels to a classifier or to keep the classifier consistent with the natural evolution of objects, shape, color, and context in time.

In the CLEAR benchmark, time-stamped Yahoo Flickr Creative Commons 100 Million (aka YFCC100m) dataset images are used. The framework utilizes CLIP for pre-labeling and MTurk for crowdsourced verification. The main goal is to have a dataset with natural time evolutions of objects within a decade (2004-2014) for 11 classes. This may allow a more realistic comparison of new ideas as previous benchmarks depend on the modifications of existing datasets (e.g. Permuted MNIST, Split-MNIST, Split-CIFAR, CORe50).

Theoretically, gradient-based learning is local in parameter space and prone to what is known as catastrophic forgetting where a decrease in test-time performance is observed concerning the previous tasks/data when the machine learning model is updated. In practice, the general heuristic in machine learning engineering is to replay the previous data within the new training iterations, however, this is not very efficient as it leads to a significant increase in training time (i.e. linearly correlated with the cumulative data points). This makes continual learning a very active research field. Recently a new research organization was also formed to enable knowledge sharing between research groups (i.e. the group maintains a common repository named Avalanche for models and data).

For more detailed information about the benchmark, the official webpage can be consulted. To discover more about continual machine learning research, the following review article can be a great source.

About the Author

Sabri Bolkar

Show moreShow less

TECH NEWS RELATED

Robots play with play dough

A new system lets robots manipulate soft, deformable material into various shapes from visual inputs, which could one day enable better home assistants.

View more: Robots play with play dough

KIT-AR: production line augmented reality startup raises £1.7m

Augmented worker platform KIT-AR has secured €2m (£1.7m) in a seed funding round co-led by Armilar Venture Partners and Caixa Capital Risc for its production line tech. Augmented reality (AR), which overlays graphics onto the real world via a screen or glasses, is being embraced by some areas of ...

View more: KIT-AR: production line augmented reality startup raises £1.7m

Computer chips powered by human brain cells already exist — but is it ethical?

How do we feel about using organic computers for cognitive labor?

View more: Computer chips powered by human brain cells already exist — but is it ethical?

Warehouse robot startup BotsAndUs raises £11m

Autonomous warehouse robot maker BotsAndUs has secured $13m (£10.6m) in a seed round led by Swiss venture capital firm Lakestar. London-based BotsAndUs will use the capital to develop its robots that are used to measure, track and locate stock in a warehouse. The startup’s robots use artificial intelligence (AI) ...

View more: Warehouse robot startup BotsAndUs raises £11m

'How to Talk to the Dead?' Alexa May Make it Possible By Mimicking the Voice of Deceased Love One

Alexa’s latest feature can mimic the sound of almost anyone, alive or dead, by learning from just a minute’s worth of audio. (Photo : Nicolas J Leclercq via Unsplash) Alexa Can Impersonate Almost Anyone’s Voices Rohit Prasad, the company’s senior vice president and head scientist for Alexa, announced a slew ...

View more: 'How to Talk to the Dead?' Alexa May Make it Possible By Mimicking the Voice of Deceased Love One

Integrated adoption of AI, data utilization can add $500 billion to India’s GDP: report

Agencies “If a petition comes, what’s there between the lines can be read through the help of artificial intelligence to see if there is something more in it than what has been stated (by the petitioner),” he said. The integrated adoption of an artificial intelligence and data utilisation strategy can ...

View more: Integrated adoption of AI, data utilization can add $500 billion to India’s GDP: report

British digital ID startup Yoti trials Instagram age verification

London-based digital identification software startup Yoti has launched a trial with Facebook-owned Instagram to test its age verification technology on the social media platform. The partnership between Yoti and Instagram will begin by testing US users who try to edit their date of birth to appear over 18. Instagram ...

View more: British digital ID startup Yoti trials Instagram age verification

UK left it too late to take golden share in Arm, says Hermann Hauser

The UK government should have taken a golden share in semiconductor designer Arm “a long time ago” and any attempt to do so now is “trying to close the gate after the horse has bolted”, according to the company’s co-founder Hermann Hauser. In an exclusive interview with UKTN, Hauser ...

View more: UK left it too late to take golden share in Arm, says Hermann Hauser

Alan Turing Birth Anniversary: What You Should Know About the Father of Modern Computer Science and Artificial Intelligence

MIT Engineers Build LEGO-Like Reconfigurable Artificial Intelligence Chip

GM’s AI Diagnostic System Vows to Make Vehicle Inspection Instantaneous — Is It Accurate?

General Motors Made an Undisclosed Strategic Investment in AI

Microsoft quits its creepy, emotion-reading A.I.

Microsoft Will No Longer Allow Companies to Access its Facial Recognition Technologies

A 3D-Printed Nixie Clock Powered By An Arduino Runs This Robot

Machine Learning Does Its Civic Duty By Spotting Roadside Litter

Microsoft Has Decided to Restrict Access to Its Facial Recognition Tools

The Power and Pitfalls of AI for US Intelligence

How to pitch your startup to an AI journalist

Researchers release open-source photorealistic simulator for autonomous driving

OTHER TECH NEWS

Top Car News Car News