Diagnosing Cross-Encoder Rerankers in Cold-Start Recommendation: Coverage, Score Signals, and Exposure Effects

Reranking with cross-encoders (including LLM-derived rerankers) is a popular design for retrieval-based recom-menders, but strict cold-start exposes a key limitation: rerankers can only reorder retrieved candidates. We present an empirical diagnosis of a CrossEncoder reranker in a cold-start movie recommendation pipeline on Serendipity-2018. In an updated evaluation over 500 cold-start users (3 seeds), a popularity baseline strongly outperforms reranking (HR@10: 0.268 vs. 0.008; nDCG@10: 0.224 vs. 0.005). Diagnostics show that (i) ground-truth items frequently sit deep in the candidate pool (median rank 6717), (ii) hybrid candidate generation yields low recall@K relative to ANN baselines, and (iii) reranker scores barely correlate with relevance (Spearman r ≈ 0), while exposure concentrates on a handful of items. To reflect the algorithm tweak that produced the new results, we also include a compact comparison to an earlier pilot run (30 users/seed) to illustrate how scaling and retrieval/candidate logic shifts observed quality. We conclude with practical mitigation steps: strengthen retrieval first, tune candidate pool size, calibrate scores, and apply exposure-aware post-processing.