acceptodds
Under review as a conference paper at ICLR 2027

PhosInv: Learning a Shared Implant Placement and Stimulus-Conditional Electrode Selection for Visual Cortical Prostheses

Abstract

Visual cortical prostheses aim to restore rudimentary vision by electrically stimulating the primary visual cortex to evoke percepts known as phosphenes. Their design requires determining where to implant a fixed electrode array and which contacts to activate for each target stimulus. Prior work reoptimises implant placement separately for each stimulus, an approach that cannot be implemented once a device has been implanted. We reformulate this task as a generalisation problem: learning a single implant placement shared across a distribution of target stimuli, together with a stimulus-conditional electrode-activation policy. PhosInv implements this formulation as a convolutional inverse model trained end-to-end through a differentiable reimplementation of an existing phosphene simulator. Once trained, it predicts a sparse electrode selection for each new stimulus in a single forward pass, without per-stimulus optimisation. On a benchmark of 2,820 targets from two domains, all scored under the differentiable renderer, domain-matched PhosInv attains lower loss than per-stimulus Bayesian placement search in all six subject hemispheres in both domains. A forward pass takes milliseconds on a GPU against minutes of CPU search per stimulus, a gap that combines amortisation with differences in hardware and implementation, and PhosInv commands six to eight times fewer contacts, reducing delivered charge, a quantity constrained by safety considerations. A model trained on an equal mixture of both domains retains most of each domain-specific model's advantage while avoiding the degradation either shows outside its own domain, indicating that training coverage, not the architecture itself, limits generalisation. Relative to a shared all-contacts-active reference, learned electrode selection contributes about three times the improvement achieved by per-stimulus placement search and far more than a placement searched once per hemisphere, identifying electrode selection as the more consequential lever for future high-density arrays. Every contrast is reported per anatomy, as an effect size and its consistency across six subject hemispheres.

Primary area:
applications to neuroscience & cognitive science
Keywords:
visual cortical prosthesis, phosphene simulation, differentiable simulation, implant placement optimization, electrode selection, amortized inference, inverse model, convolutional neural network, neural prosthetics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qfkkOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents PhosInv, a method that jointly optimizes electrode placement and stimulation patterns for visual cortical prostheses. This work differentiably reimplements a prior phosphene simulator, enabling end-to-end optimization of both placement parameters and a conditional selection network. Based on this simulator, PhosInv achieves higher predicted quality than per-target Bayesian optimization baselines.

Critical strengths

This work try to connect the previously separate problems of electrode placement and selection, bringing the visual cortical prosthesis closer to realistic device constraints. The differentiable simulator may also support further research on electrode implantation and selection strategies.

Critical weaknesses

The experimental results are limited: (1) PhosInv is trained with access to the simulator, potentially biasing the comparison against BO methods. (2) The realism of the differentiable simulator is unverified, yet all main results depend on it. (3) Whether the jointly optimized placements are individually optimal is not assessed.

Overall, only Table 1 is presented in the main text, providing insufficient support for the conclusions.

Questions for authors
  1. Does PhosInv’s reliance on the simulator during training cause overfitting, making the comparison against BO unfair when both are evaluated on the same simulator?

  2. How is the biological realism of the modified simulator validated?

  3. Could BO be optimized on the differentiable simulator before comparison, or could more baselines be added? This would allow a more systematic evaluation of PhosInv.

  4. Are the jointly optimized placements and selections also individually optimal? If not, what specific improvement does joint optimization bring to each component?

  5. The paper compares PhosInv on GPU against BO on CPU. Has a GPU-based BO been tested? Such a comparison would be fairer.

3 comments
AuthorsOpenReview ↗

Response to Reviewer qfkk

Thank you very much for your review. Your first question turned out to be the most important one for this paper. The overfitting you suspected was real. We found it, traced its cause, and revised the method on that basis rather than defend the submitted joint-learning result. The table summarises what we did for each of your concerns, and short answers follow.

Your concernWhat we didAnswer
Q1. Overfitting from simulator accessSame reference simulator for all methods; renderer ladderThe confound was real; we found it and removed it
Q2. Biological realismDropped the modified simulator; stated as a limitationWe do not claim biological validity; we removed simulator mismatch as a comparative confound
Q3. BO on the same simulator, more baselines14 methods and a published encoder (HNA)Learned selection beats BO in 20/20 hemispheres, and not only with our CNN
Q4. Placement versus selection2x2 lever decompositionSelection 0.28 to 0.32; per-target placement 0.02 to 0.05
Q5. GPU fairnessBO objective ported to the same GPU121 s per target; 99% is the GP optimiser

Q1 (overfitting from simulator access). In the submitted version the model was trained through a smoothed reimplementation that averaged pRF properties over nearby voxels. With this averaging about 995 contacts per placement count as valid instead of about 462, and the jointly learned model exploited it. Rescored under the reference simulator, it is worse than BO in all six hemispheres (Sec. 6.5, Fig. 3). We therefore no longer learn the placement through the smoothed renderer. All methods are now fitted and scored under the same reference simulator, on held-out targets, and the selection network is trained through that simulator, which at a fixed placement is an exact sum of single-contact images.

Q2 (biological realism). We do not claim biological validation of the simulator. Our revision instead removes simulator mismatch as a comparative confound: all methods are now fitted and evaluated under the same published reference simulator of van Hoof et al. (2025), without modification. Biological validity remains an explicit limitation (Sec. 7).

Q3 (BO on the same simulator and more baselines). Under one reference simulator, learned selection beats per-stimulus BO in 20/20 hemispheres of 10 subjects (letter -0.140, geometric -0.138; Sec. 6.1, Table 1), including the 10 left hemispheres, where both methods search identical placement bounds. The benchmark compares 14 methods. We also trained a published learned stimulus encoder (HNA, Granley et al., 2022) in place of our CNN on the six original hemispheres. It matches our CNN and beats BO in 6/6 (Appendix E, Table 8), so the result does not depend on our encoder.

Q4 (whether placement and selection are each optimal). A 2x2 analysis separates the two. Once contacts are selected per target, refitting the placement per target improves the loss by only 0.02 to 0.05, while selection improves it by 0.28 to 0.32 (Sec. 6.3, Appendix I). For selection, a per-target mask search at the shared placement serves as a strong search reference (not a guaranteed optimum). Our network is within 0.052 of it on letters and 0.002 on geometric targets.

Q5 (GPU-based BO). We ported the whole BO objective to the same GPU, verified it against the original code to within 4e-8, and reran BO on 72 targets. A search still takes 121 s per target, because 99% of the time is spent in the Gaussian-process optimiser and not in the simulator. Our forward pass remains about 30,000 times faster on the same GPU (Sec. 6.4, Appendix K).

Main text containing only Table 1. The main text now also has three figures: the method overview (Fig. 1), per-hemisphere differences from BO (Fig. 2) and the renderer ladder (Fig. 3).

We hope these experiments address your concerns, and we would be glad to run any further experiment you consider necessary.

I thank the authors for the additional experiments, which ensure a fairer comparison by employing the same simulator. However, fixing the the simulator undermines the paper's core innovation: the joint optimization of placement and selection. Therefore, I maintain my original score.

AuthorsOpenReview ↗

Reply to Reviewer qfkk

Thank you very much for reading our response and for acknowledging that the comparison is now fairer. We would like to clarify one point, because we believe it concerns how the change in the revision is described rather than the evidence itself.

The revision does not alter the simulator; the submitted version did. To make placement differentiable, the submitted version trained through a smoothed reimplementation that averaged pRF properties over nearby voxels. The revision uses the reference simulator of van Hoof et al. (2025) as published, without modification. Under that simulator, the gains of the jointly learned placement reverse (Sec. 6.5, Fig. 3).

The joint problem is unchanged; only the solver changed. Eq. 3 still optimises one shared placement and one stimulus-conditional selection policy for a single objective over the target distribution. We regard this formulation, rather than any particular solver, as the core of the paper. The submitted solver, gradient descent through the smoothed surrogate, learned gains that do not exist in the original simulator. The revised solver addresses the same problem under the original simulator: it searches the placement once per hemisphere and learns the selection through the exact renderer.

An additional result suggests that the two variables remain coupled under the original simulator. We repeated the placement search, scoring each candidate placement with per-target selection in the loop rather than with all contacts active. Under per-target mask search at the shared placement, the placements found this way are better than the placement used in the paper in 19 of 20 hemispheres (letter −0.066, geometric −0.059; seed SD 0.008). This suggests that the shared placement should be chosen for the selection policy, and that this can be done without smoothing the simulator. We will add this experiment to the final version.

We hope this clarifies that the revision keeps the joint formulation and changes only the way it is solved. Thank you again for your time and for the questions that led to these experiments.

Reviewer_MHxyOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents PhosInv, a framework for jointly learning a subject-specific implant placement and a stimulus-conditional electrode-selection policy for visual cortical prostheses. The main motivation is that an implant placement must remain fixed after surgery, whereas previous placement-search methods optimize a different geometry for each target stimulus. PhosInv instead learns one placement shared across a distribution of targets and uses a convolutional inverse model to select a sparse subset of electrodes for each new target.

The model is trained end-to-end through a differentiable PyTorch reimplementation of an existing phosphene simulator. The placement is represented by four learned parameters shared across all training samples for a subject hemisphere, while the encoder predicts 1,000 electrode-activation logits for each stimulus. The paper evaluates domain-specific and mixed-domain models on 2,820 letter-like and geometric targets across three subjects and both hemispheres. Under the differentiable renderer used for evaluation, the domain-matched PhosInv models achieve lower composite loss than per-target Bayesian placement search, use substantially fewer active contacts, and replace per-target optimization with a forward pass. Additional experiments analyze cross-domain transfer, the relative contributions of placement and electrode selection, inference cost, and robustness across anatomical units.

Critical strengths
  1. The paper identifies an important mismatch between previous optimization formulations and the physical constraints of an implanted device. A cortical implant cannot be repositioned for every visual target, so replacing per-target placement optimization with one anatomy-specific placement and a stimulus-conditional activation policy is a practically meaningful reformulation. The resulting distinction between subject-level and stimulus-level decision variables is clear and potentially useful beyond the particular architecture used here.

  2. The paper is unusually transparent about negative results and limitations. It reports effects separately across six subject hemispheres, avoids relying heavily on underpowered p-values, acknowledges the moderate disagreement between the two simulator implementations, retracts an earlier yield-related interpretation that did not survive recomputation, and explicitly states that joint placement learning is not shown to outperform a two-stage alternative. This transparency makes the remaining empirical evidence easier to interpret.

Critical weaknesses
  1. The main comparison is confounded by a substantial renderer mismatch. PhosInv is trained directly through the differentiable renderer used for final scoring, whereas the Bayesian-search baselines optimize their placements using the reference NumPy renderer. The paper reports only moderate agreement between the two implementations and explicitly acknowledges that a method is expected to perform better under the renderer to which it was fitted. Consequently, the headline result that PhosInv outperforms per-target Bayesian search under the differentiable renderer does not establish that PhosInv is better under a common forward model. It may partly measure renderer alignment rather than the proposed optimization formulation. A fair evaluation would require rerunning Bayesian search against the differentiable renderer, or fully scoring configurations under both renderers after fitting each method consistently.

  2. The experiments do not isolate the value of jointly learning implant placement, and the strongest missing baseline is close to the method's own identified operating point. The paper finds that most of the gain comes from learned electrode selection rather than placement, and the learned placement is not consistently better than a placement searched once per hemisphere. However, it does not evaluate a learned stimulus-conditional selector on top of a fixed placement obtained by a once-per-hemisphere search. It also does not compare against stronger existing learned stimulus encoders for fixed device geometries. Without these controls, the results support learned electrode selection but do not establish that moving placement into the end-to-end learned problem is necessary or beneficial. This substantially narrows the demonstrated contribution relative to the paper's joint-optimization framing.

Questions for authors
  1. Can the authors rerun the Bayesian-search and related search-based baselines directly against the differentiable renderer used to train and evaluate PhosInv, using the same placement bounds and loss definition? Alternatively, can all methods be fitted and evaluated under both renderer implementations? Given the reported moderate agreement between the renderers, this result is necessary to determine whether the headline advantage reflects the proposed formulation or renderer alignment.

  2. What is the performance of the stimulus-conditional PhosInv selection network when the implant placement is fixed to a placement searched once per subject hemisphere? The current decomposition indicates that selection produces most of the improvement and that the jointly learned placement is not consistently better than a search-once placement. This missing two-stage baseline would materially affect whether joint optimization of placement and selection is a supported contribution.

3 comments
AuthorsOpenReview ↗

Response to Reviewer MHxy

Thank you very much for your thoughtful review. Your suspicion about renderer alignment was correct, and running the two experiments you asked for changed our conclusion and our method.

W1/Q1 (common renderer). All methods are now fitted to and scored under the same reference simulator, the original code of van Hoof et al. (2025). The renderer ladder (Sec. 6.5, Fig. 3) is the "score under both renderers" experiment you suggested, with every intermediate step added. Under the reference simulator the jointly learned model is worse than BO in all six hemispheres. The reversal happens at a single step, where voxel-averaged pRF properties are replaced by each voxel's own (L3). We therefore withdraw the claim that joint learning beats search.

W2/Q2 (two-stage baseline). The two-stage approach is now our method. The placement is searched once per hemisphere on the reference objective, and the selection network is trained on it through the reference simulator, which at a fixed placement is an exact and differentiable sum of single-contact images. It beats per-stimulus BO in 20/20 hemispheres of 10 subjects (letter -0.140, geometric -0.138), including the 10 left hemispheres, where both methods search identical bounds. The searched placement is also better than the jointly learned one in 15 of 18 hemisphere-domain pairs (Appendix N). Your reading that the gain comes from selection is confirmed: once contacts are selected, refitting the placement per target adds only 0.02 to 0.05, compared with 0.28 to 0.32 from selection (Sec. 6.3).

W2 (existing learned stimulus encoders). We replaced our CNN with the stimulus encoder of the hybrid neural autoencoder of Granley et al. (2022) and kept everything else the same, including the placements, renderer, loss, data split and schedule. Over three seeds on each of the six original hemispheres, it matches our CNN (letter 1.772 vs 1.800, better in 5/6; geometric 1.684 vs 1.677, better in 4/6) and beats BO in 6/6 hemispheres in both domains (Sec. 6.3, Appendix E, Table 8). On these hemispheres the selection result therefore does not depend on our particular encoder.

We agree that the contribution is narrower than the submitted framing, and the revised paper states it as such. It is not a new encoder architecture, but an amortised formulation of stimulus-conditional electrode selection under a physically fixed implant. Alongside it we report an exact differentiable renderer obtained from the original simulator, a decomposition showing that selection is the main lever in this setting, and a documented failure case of learning the placement through a smoothed surrogate, with its cause identified. The formulation, data, benchmark, simulator and loss are unchanged. We would be grateful if you could take the revised results into account.

Thank you for the additional experiments and clarification. The revised results substantially address my concerns about renderer alignment and the missing baselines. However, they support a narrower contribution centered on electrode selection rather than joint placement learning. Balancing the stronger evidence against this narrower contribution, I maintain my rating of weak acceptance.

AuthorsOpenReview ↗

Reply to Reviewer MHxy

Thank you for reconsidering the revised results and for your constructive review throughout. We agree that the supported contribution is centred on stimulus-conditional selection under a fixed implant, and the revised paper is framed accordingly.

Reviewer_XZENOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PhosInv, which jointly learns a fixed implant placement and a stimulus-dependent electrode-selection policy through a differentiable phosphene simulator. Experiments across two target domains and six subject hemispheres show improved simulated reconstruction quality and faster inference than placement-search baselines.

Critical strengths
  1. Separating a placement shared across stimuli from a stimulus-dependent activation policy seems to be an important constraint when deployed in realistic.

  2. The domain-matched models outperform BO in mean reconstruction loss across all six subject hemispheres in both domains.

  3. The discussion about the paper's limitation, e.g., the renderer mismatch, the absence of a complete two-stage baseline, the limited number of subjects, and the hardware differences in timing measurements, is sufficient.

Critical weaknesses
  1. BO is optimized using a different simulator from the one used for evaluation, while PhosInv is trained directly on the evaluation simulator.

  2. Without training the same selection network on a previously searched, fixed placement, the benefit of jointly learning placement remains unclear.

  3. Experiments cover two target domains, three subjects, and one training seed, providing limited support for broader claims about generalization and the relative importance of selection versus placement.

Questions for authors
  1. Does PhosInv retain its advantage when BO optimizes against the same simulator used for evaluation?

  2. How does joint optimization compare with training the identical selection network on a placement obtained beforehand by search?

  3. Table 2 indicates different right-hemisphere β\beta ranges for the learned and search-based methods. Do these ranges represent equivalent physical configurations under different coordinate conventions?

2 comments
AuthorsOpenReview ↗

Response to Reviewer XZEN

Thank you very much for your review. We answer each of your questions briefly below.

Q1 (BO on the evaluation simulator). All methods are now fitted to and scored under the same reference simulator. Learned selection beats BO in 20/20 hemispheres of 10 subjects (letter -0.140, geometric -0.138; Sec. 6.1). The jointly learned placement did not keep its advantage, and we withdraw that claim (Sec. 6.5).

Q2 (two-stage). Training the selection network on a placement found beforehand by search is now the method. The searched placement is better than the jointly learned one in 15 of 18 hemisphere-domain pairs (Appendix N), so learning the placement jointly brings no benefit.

Q3 (right-hemisphere beta ranges). You are right that the ranges are not equivalent. In right hemispheres per-stimulus BO searches beta in [-110, 15] degrees, as in the original code, while our placement search uses [-15, 110]. The paper now states this (Sec. 5.3). We therefore treat the 10 left hemispheres, where both methods search identical bounds, as the fairness check and report them directly after the headline result. Learned selection beats BO in all 10 of them (geometric -0.147, letter -0.120; Sec. 6.1, filled markers in Fig. 2).

W3 (scope). We extended the study from 3 subjects (6 hemispheres) to 10 subjects (20 hemispheres). Each configuration is trained with five seeds on the original hemispheres and three on the added ones, and the result holds for every seed. Having only two target domains remains a limitation, which we state in Sec. 7.

Ethics flag. The study uses only the public, de-identified HCP 7T dataset under its terms of use and is entirely computational. No experiments on humans or animals were performed, and we make no clinical claims. The paper also notes that the pipeline needs retinotopy data that blind recipients cannot provide (Sec. 7). We would be happy to add anything the AC finds necessary.

AuthorsOpenReview ↗

Reply to Reviewer XZEN

Thank you again for your review. As the discussion period closes soon, we would like to check whether our response addressed your questions: BO refitted under the same simulator (Q1), the two-stage comparison (Q2) and the right-hemisphere beta ranges (Q3). We would be glad to clarify anything that remains unclear.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: what changed in the revision and why

We sincerely thank the reviewers for their careful and constructive reviews. All three reviews asked for the same two experiments: to refit and score every method under one simulator, and to train the same selection network on a placement found beforehand by search. We ran both. They changed our main conclusion, and we revised the method as the reviewers' experiments dictated rather than defend the submitted joint-learning result.

What changed.

VersionCore formulation
SubmittedJointly learn the shared placement and the stimulus-conditional selection through a smoothed differentiable surrogate of the simulator.
RevisedSearch the shared placement once per hemisphere on the reference simulator, and learn only the stimulus-conditional selection, through the reference simulator itself, which is exactly differentiable at a fixed placement.

What did not change. The problem formulation (one placement shared across a target distribution, with a stimulus-conditional selection), the data, the 2,820-target benchmark, the simulator and the loss are those of the submission. We kept the section structure and the layout of Table 1 so that the revision can be read against the submitted version.

Why we changed it. When every method is scored under the reference simulator of van Hoof et al. (2025), the jointly learned model is worse than per-stimulus BO in all six hemispheres, and we withdraw the claim that joint learning beats search. To find the cause, we replaced the smoothed approximations of our reimplementation one at a time (Fig. 3, L0 to L6). The reversal happens at a single step (L3). The smoothed renderer averages pRF properties over nearby voxels, so about 995 contacts count as valid instead of about 462, and the jointly learned model had learned to exploit this.

Results with one simulator for all methods.

  1. Learned selection beats per-stimulus BO in 20/20 hemispheres of 10 subjects (letter -0.140, geometric -0.138), for every training seed. The same holds in the 10 left hemispheres alone, where both methods search identical placement bounds (10/10; letter -0.120, geometric -0.147; Sec. 6.1, Fig. 2).
  2. Selection is the main lever in this setting. Once contacts are selected per target, moving the array per target improves the loss by only 0.02 to 0.05, compared with 0.28 to 0.32 from selection (Sec. 6.3).
  3. The result does not depend on our encoder. On the six original hemispheres, the published HNA encoder (Granley et al., 2022), trained in the same way, matches our CNN and also beats BO in 6/6 (Appendix E, Table 8).
  4. BO remains slow on a GPU. With its whole objective ported to the same GPU, a search still takes 121 s per target, because 99% of the time is spent in the Gaussian-process optimiser rather than in the simulator (Sec. 6.4).

Contribution. The contribution is narrower than the submitted framing, but substantially better controlled. It is not a new encoder architecture, but an amortised formulation of stimulus-conditional electrode selection under a physically fixed implant. The gain over per-stimulus search comes from selecting contacts per stimulus on the one placement a device can keep, and amortisation delivers that gain in a single forward pass, within 0.002 to 0.052 of a per-target mask search. A forward pass takes a few milliseconds against minutes of search, which is compatible with real-time inference. The lever analysis suggests that selection may be a more consequential optimisation target than placement precision in this setting. Finally, the renderer ladder documents how a model trained through a smoothed simulator can learn gains that do not exist in the original one.

Where to find the changes. Sec. 3 (two-stage solution), Sec. 4.1 (exact renderer), Sec. 5.1 (10 subjects, 20 hemispheres), Sec. 6.1 to 6.5 (all results rerun under the reference simulator, with GPU BO in 6.4 and the renderer ladder in 6.5), Appendix E (HNA, Table 8) and Appendices K, M and N.

Because the title cannot be changed during the discussion period, we keep it for now. If the paper is accepted, we would like to retitle it PhosInv: Searching a Shared Implant Placement and Learning Stimulus-Conditional Electrode Selection for Visual Cortical Prostheses to match the revised method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.