- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This manuscript proposes a path towards personalization in MCP-based tool calling. The authors formulate the problem as a ranking task where a user may have preference priors for some functions over others. To analyze this, they construct a sparse user–tool matrix that is factorized to produce low-dimensional embeddings used for retrieval or preference alignment.Three dimensionality-reduction approaches are presented: a shared autoencoder with a weighted reconstruction loss (Algorithm 2), truncated SVD with user factors (Algorithm 3), plus standardized PCA and raw tool-count vectors as baselines.
The embeddings are meant to support user similarity (cosine/Euclidean), cohort discovery via k-means, and downstream "personalization features" such as prompt suggestions and customized responses — though no concrete mechanism for feeding the embeddings back into tool ranking is specified.
The authors design a new dataset for this purpose and conduct a straightforward analysis on “canary” users, split into two groups biased toward having centralized and sparse embedding-based clusters, respectively.
Critical strengths
1\ Timely and well-motivated problem framing. Recasting MCP tool invocations as RecSys via user–item matrix where "items" are tools is a novel and sound exploration with privacy-first design at hand.
2\ Reproducibility artifacts: The authors provide a public synthetic dataset, generator code, and the canary-based evaluation protocol that allow readers to reproduce the experiments failrly easily.
Critical weaknesses
While the authors provide an interesting problem setting, there are several key challenges with the paper itself:
1\ It is not entirely clear how the authors intend for these new embeddings to be used in tool-calling ranking.
2\ The technical novelty of the paper is fairly limited to the application domain.
3\ An interesting and potentially more powerful baseline would be to use the mean or median of the embeddings from the textual descriptions of the tools called by a user's LLM.
4\ The rich semantic metadata that MCP tools carry (descriptions, input schemas) is overlooked in favor of counts alone, even though the paper's own Section 1.1 states that tool selection is driven by semantic similarity over descriptions.
5\ It is not entirely clear why the cold-start problem would be a major bottleneck here, since tool calling is naturally a semantically rich process: textual descriptions of both the task and the functionality of each tool are available at all times.
Questions for authors
Please review points in Critical Weaknesses.
Any remaining questions ?
Dear Reviewer ZKm4, thank you again for your thoughtful review. As the discussion period progresses, we wanted to check whether our response and addresses your concerns, in particular your concerns on synthetic data and downstream evidence?