With the increasingly impressive developments in machine learning and consequently, deep learning algorithms, we are seeing amazing new kinds of data modeling that is producing better-tailored results for a wider range of specific data processing requirements. With the help of DevOps and MLOps, deep learning has furthered its reach in various kinds of individuals and industries alike. From bespoke IT services to containerization through docker-based applications (you can read up OpenShift vs. Kubernetes to see how they can help your management processes), there have been massive improvements in both personal and business fronts! Nonetheless, technology has also been branching out to produce new solutions from these data models. Therefore, I am going to show you how some intelligent people are tricking deep learning algorithms into doing new things.
Pretrained and Finetuned Deep Learning Models
When you want to create an application that requires deep learning technologies, one option is to make your own neural network from the ground up and train it on available or curated examples. For example, you can use sites like ImageNet, a public dataset that contains more than 14 million labeled images.
There is a problem, however. First, you have to find the right architecture for the task, the number of sequences of convolution, pooling, and dense layers should be taken into account. You have to also decide the number of filters and parameters for each layer, the learning rate, optimizer, loss function, and other hyperparameters.
Many of these choices require tons of trial-and-error training, which is time and money-consuming unless you have access to some powerful graphics processors or specialized hardware built for these workloads such as Google’s TPU.
To avoid reinventing the wheel, you can download a tried-and-tested model like AlexNet, ResNet, or Inception, and train it to cater to your specific problem. But that does not mean you will not need a cluster of GPUs or TPUs to complete the training in a reasonable time frame. To avoid unnecessary costs during the training process, you can download the pre-trained version of these models to integrate them into your application instead.
Adversarial Attacks and Reprogramming
Adversarial reprogramming is an alternative approach for repurposing machine learning models. It takes advantage of adversarial machine learning, an area of research that explores how perturbations to input data can change the behavior of your neural networks. For example, you could add a layer of noise to a photo of a panda and it will probably cause the award-winning GoogleNet deep learning model to mistake it for a gibbon. The manipulations are referred to as “adversarial perturbations.”
Adversarial machine learning is usually used to display vulnerabilities in deep neural networks. Researchers in the field, often use the phrase “adversarial attacks” when talking about machine learning. One of the main aspects of adversarial attacks is that the perturbations must go undetected by the human eye. Moreover, machine learning combined with blockchain technology can also be used to detect web3 security threats, a service that can be provided by companies similar to Luabase to various IT businesses. Such tech upgrades can help businesses to achieve new milestones and automate more IT functions.
Black-box Adversarial Learning
Adversarial reprogramming does not modify the original deep learning model, you still need access to the neural network’s parameters and layers in order to train and tune the adversarial program (with focus on access to the gradient information). This suggests that you cannot apply it to black-box models.
This is where black-box adversarial reprogramming (BAR) enters the picture. The adversarial reprogramming method developed by top researchers from establishments like IBM and Tsing Hua University does not need access to the details of deep learning models to alter their behavior.
To achieve this, the researchers used Zeroth Order Optimization (ZOO), a method formally developed by AI researchers at IBM and the University of California Davis. The ZOO paper proved the feasibility of black-box adversarial attacks, where an attacker could manipulate the behavior of a machine learning model by deciding to watch inputs and outputs and without having access to the gradient information.
BAR uses the same methods to train the adversarial program. To test black-box adversarial reprogramming, the researchers used it to repurpose several popular deep learning models for three medical imaging tasks (diabetic retinopathy detection, autism spectrum disorder classification, and melanoma detection). Medical imaging is an incredibly attractive use for approaches such as BAR because it is a domain where data is scarce, expensive to come by, and subject to privacy regulations.
To conclude, deep learning models still have a long way to go but the current results are promising. Future AI researchers will further explore how BAR can be applied to a wider range of data modalities beyond image-based applications. Perhaps one day, these methods can be applied to almost every R&D issue that any company could face to better improve solutions through this kind of idea “creation”.
