RASALoRE: Region-Aware Spatial Attention with Location-based Random Embeddings for Weakly Supervised Anomaly Detection in Brain MRI Scans

British Machine Vision Conference (BMVC) 2025
Bheeshm Sharma1, Karthikeyan Jaganathan2, Balamurugan Palaniappan3
1,3Department of Industrial Engineering & Operations Research (IEOR), IIT Bombay, India
2Department of Energy Science and Engineering (DESE), IIT Bombay, India
📄 Paper 💻 Code 📊 Poster
RASALoRE Teaser
Figure 1: RASALoRE achieves state-of-the-art weakly supervised anomaly detection in brain MRI scans using only slice-level labels, significantly outperforming existing methods while using less than 8M parameters.

Abstract

Weakly Supervised Anomaly detection (WSAD) in brain MRI scans is an important challenge useful to obtain quick and accurate detection of brain anomalies when precise pixel-level anomaly annotations are unavailable and only weak labels (e.g., slice-level) are available. In this work, we propose RASALoRE: Region Aware Spatial Attention with Location-based Random Embeddings, a novel two-stage WSAD framework. In the first stage, we introduce a Discriminative Dual Prompt Tuning (DDPT) mechanism that generates high-quality pseudo weak masks based on slice-level labels, serving as coarse localization cues. In the second stage, we propose a segmentation network with a Region-Aware Spatial Attention(RASA) mechanism that relies on fixed Location-based Random Embeddings. This design enables the model to effectively focus on anomalous regions. Our approach achieves state-of-the-art anomaly detection performance, significantly outperforming existing WSAD methods while utilizing less than 8 million parameters. Extensive evaluations on the BraTS20, BraTS21, BraTS23, and MSD datasets demonstrate a substantial performance improvement coupled with a significant reduction in computational complexity.

Key Contributions

  • A novel two-stage weakly supervised anomaly detection (WSAD) framework that operates using only slice-level labels.
  • A prompt-tuning strategy (DDPT) to generate high-quality pseudo-masks from slice level supervision using vision-language models.
  • A Region-Aware Spatial Attention (RASA) mechanism guided by Location-based Random Embeddings (LoRE) to effectively capture local contextual dependencies.
  • Superior performance while significantly reducing the number of model parameters (~7.8M) compared to existing methods.

Method Overview

Our approach consists of two stages: (1) Discriminative Dual Prompt Tuning (DDPT) for generating pseudo weak masks, and (2) RASALoRE segmentation network for precise anomaly localization.

Stage 1: Discriminative Dual Prompt Tuning (DDPT)

DDPT Architecture
Figure 2: Overview of Discriminative Dual Prompt Tuning (DDPT). DDPT employs learnable text and visual prompts with frozen encoders to classify MRI slices and generate coarse anomaly segmentation maps.

DDPT employs a classification-driven approach to generate coarse anomaly segmentation maps using only weak (slice-level) supervision. By training a discriminative network to classify brain MRI scan images, we extract attention maps that contain potential region localization information.

Stage 2: RASALoRE Architecture

RASALoRE Architecture
Figure 3: Overview of RASALoRE Architecture. The network utilizes fixed Location-based Random Embeddings (LoRE) within a Region-Aware Spatial Attention (RASA) module for precise anomaly localization.

Location-based Random Embeddings (LoRE): We generate a grid of evenly spaced Candidate Prompt Points (CPPs) across the input image. Each CPP receives fixed, non-learnable random embeddings based on sinusoidal transformations, independent of dataset-specific biases.

LoRE and Refiner
Figure 4: (a) Left: Candidate prompt point locations (in blue) overlaid as grid on input image, center: point activation mask (red denoting active and blue denoting inactive points) overlaid on input image, right: weak anomaly mask corresponding to input image. (b) Refiner Module.

Region-Aware Spatial Attention (RASA): RASA enables interaction between location embeddings and spatial information through multi-head attention, where CPP embeddings form the query, and refined image representations provide key and value components. This allows the model to effectively focus on anomalous regions.

Results

Qualitative Comparison
Figure 5: Qualitative comparison of predicted anomaly masks. RASALoRE produces sharper boundaries and more accurate anomaly localization compared to baseline methods.

Visual analysis reveals that RASALoRE achieves sharp boundaries and accurate localization across diverse tumor morphologies, significantly outperforming reconstruction-based and CAM-based approaches on BraTS20, BraTS21, BraTS23, and MSD datasets.

Key Features

  • Parameter Efficiency: Less than 8 million parameters (~7.8M)
  • Weakly Supervised: Operates using only slice-level labels
  • Fixed Random Embeddings: Non-learnable location-based embeddings independent of dataset biases
  • Region-Aware Attention: Effectively captures local contextual dependencies
  • Multimodal Extension: Successfully extends to multiple MRI modalities (T1, T1ce, FLAIR)

Acknowledgments

We gratefully acknowledge Technocraft Centre of Applied Artificial Intelligence (TCAAI), IIT Bombay for their support through generous funding.