Scat singing requires a high level of vocal control, flexibility, and creativity. Performers use a range of techniques to create their scat melodies, including:
A few recent works have explored hybrid designs that fuse the kernel‑based linearization of Performer with the block‑sparse pattern of SCAT: perverformer scat
class PerformerSCAT(torch.nn.Module): def __init__(self, dim, heads=8, seq_len=4096, block_size=512): super().__init__() self.performer = Performer( dim=dim, heads=heads, causal=True, nb_features=256, # random-feature dimension feature_type='exp' # approximates softmax ) self.scat = SparseCausalAttention( block_size=block_size, # local sliding window global_num=4 # a few global tokens per layer ) self.norm = torch.nn.LayerNorm(dim) Scat singing requires a high level of vocal
# 2️⃣ SCAT sparse causal mask on top x = self.scat(x) + x perverformer scat