acceptodds
Under review as a conference paper at ICLR 2027

MATEL: Multi-Arm Treatment Effect Learner for E-commerce Packaging Optimization

Abstract

Automated packaging selection at e-commerce scale assigns a pack type to every customer order, yet these decisions remain largely heuristic and cannot reason jointly about competing objectives: damage protection, material waste, labor, and transportation efficiency. We frame packaging selection as a multi-armed causal optimization problem and introduce the Multi-Arm Treatment Effect Learner (MATEL), a causal deep-learning algorithm that estimates the effect of each pack type on per-shipment damage rates from large-scale fulfillment data. MATEL advances causal effect estimation on four fronts: (1) a multi-armed causal prediction engine trained with self-calibrating loss and hybrid representation layers; (2) interchangeable state-of-the-art tabular backbones; (3) robustness to treatment sparsity, via renormalized pairwise-propensity weighting that keeps counterfactual contrasts bounded and the pairwise calibration doubly robust under overlap; and (4) transitive pairwise contrasts with bounded extrapolations for rarely-used pack types. Empirically, MATEL recovers the directional agreement of the observed pairwise treatment effect on 91% of focal pairs versus 62% for a double machine learning (DML) benchmark, while improving per-unit CATE directional agreement by pp. Its estimation keeps every per-unit effect bounded on the sparse arms where benchmarks diverge. BLP slope, GATES, and the doubly robust curve show MATEL is better calibrated for effect ranking than the benchmark. On synthetic data, MATEL achieves the lowest ATE error and best ranking against four benchmarks, and the lowest PEHE under weak and severe overlap. MATEL is deployed in production; a production-calibrated simulation estimates a pp network-cost improvement and a 25% reduction in corrugated-box usage, reducing packaging waste and associated carbon emissions.

Primary area:
causal reasoning
Keywords:
Causal Deep Learning, Multi-Treatment Effect Estimation, Heterogeneous Treatment Effect Estimation, Representation Learning for Treatment Effect Estimation, Treatment Sparsity and Weak Overlap, Targeted Regularization, Causal Decision-Making, Counterfactual Prediction, Uplift Modeling

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_H43DOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper tackles the challenging problem of causal effect estimation in industrial decision-making, specifically dealing with multi-arm treatments, heavy-tailed and zero-inflated outcomes, and severely sparse treatment overlap in the context of e-commerce fulfillment packaging assignment, by proposing MATEL, an end-to-end causal deep-learning framework. Its key advantages lie in its hybrid architecture (combining a tabular backbone with Tweedie outcome heads and a decoupled propensity encoder) and its theoretical pairwise renormalization, which ensures bounded estimates even under extreme sparsity, alongside a non-negative least-squares program that enforces transitive counterfactual levels for a cost-minimization engine. The industrial grounding and empirical results are highly impressive: validated on a massive dataset of 2.5 billion shipments, it achieves 91% directional agreement with observational references and recovers 91%-92% of oracle uplift in synthetic tests, while a production-calibrated simulation reports a 3.74 percentage-point network cost reduction and a 25% cut in corrugated box usage. However, the paper has notable limitations: the theoretical novelty is largely incremental, relying on engineering adaptations of existing methods (e.g., TMLE, Tweedie loss, and NNLS), and the causal assumptions are overly strict, as it strictly assumes unconfoundedness and lacks any sensitivity analysis for hidden confounders, which are highly prevalent in real-world logistics. Overall, despite these theoretical and assumption-related shortcomings, this is a solid and highly practical industrial paper with strong real-world impact.

Critical strengths

The most compelling strength of this paper is its exceptional success in translating causal machine learning into a deployed, large-scale industrial solution that solves real-world pain points. The methodological design is highly effective, elegantly tackling the notorious "sparse overlap" and heavy-tailed data issues that routinely cause traditional estimators (like IPW/DML) to fail in real-world logistics. By introducing pairwise renormalization, the framework successfully provides bounded estimates even under extreme sparsity, ensuring reliable performance on long-tail data. More impressively, it bridges the gap between causal estimation and operational execution by coupling directly with a constrained packaging optimization engine. Validated on a massive scale of 2.5 billion shipments, it delivers a tangible 3.74 percentage-point cost reduction and a 25% cut in corrugated box usage. This high-impact deployment substantiates the practical utility of the approach and provides a highly valuable blueprint for applying causal inference to complex, messy industrial decision-making systems.

Critical weaknesses

While the paper demonstrates impressive industrial impact, it has several critical weaknesses. First, the causal assumptions are overly strict: it strictly assumes unconfoundedness and lacks any sensitivity analysis for hidden confounders, which are highly prevalent in real-world logistics (e.g., unrecorded operator judgment), potentially invalidating the claimed causal estimates. Second, the theoretical contribution is largely incremental, combining existing tools like Tweedie regression and TMLE, yet the paper presents a dense and cluttered array of complex mathematical formulas that obscures the core message. This lack of clarity makes it difficult to follow; since the primary strength lies in practical deployment, the authors should significantly reduce the overwhelming theoretical notation and instead focus more on clearly articulating the real-world engineering insights, the specific challenges of the logistics data, and how the proposed solution actually addresses them.

Questions for authors

I have a few questions and suggestions that could materially affect my assessment. To clarify the robustness of your claims, given that packaging assignment is highly likely driven by unrecorded factors (e.g., operator intuition, unobserved transit conditions), how robust are the reported production gains if hidden confounding exists, and can you provide any sensitivity analysis to quantify this risk? Regarding the empirical validation, since the real-world evaluation uses the model's own AIPW pseudo-outcomes as the reference for BLP and GATES diagnostics, could this self-referential evaluation potentially inflate MATEL's performance compared to the baselines? Furthermore, the synthetic data-generating process appears specifically tailored to favor the proposed pairwise renormalization; could you discuss how the method performs under different, less favorable data-generating distributions? Finally, as a recommendation for improving the manuscript's focus, given that the theoretical novelty is largely incremental (combining existing tools like TMLE and Tweedie), I strongly encourage the authors to streamline the dense array of mathematical formulas and instead dedicate more space to clearly articulating the practical engineering insights, the specific characteristics of the real-world data distribution, and the deployment pipeline, as the true impact of this work lies in its industrial application.

6 comments
AuthorsOpenReview ↗

Rebuttal response to reviewer H43D

Thank you for your thoughtful review and for recognizing the translation of causal machine learning into a deployed, large-scale decision system. We sincerely appreciate the time and effort you devoted to our paper.

Q1. Given that packaging assignment is likely driven by unrecorded factors, provide a sensitivity analysis to quantify this risk.

We agree that unrecorded factors can threaten identification, and we added three analyses to assess this risk.

First, we use the simulation of Section 4.3 as a sensitivity test for biased damage estimates. A hidden factor could make MATEL under- or over-predict damage, which the simulation reproduces by scaling the damage weight from 0 to 10 (Table D.2). Across this range, the saving stays positive at 1.94–3.74 pp, and the 2.87 pp realized in production falls inside it.

Second, data from the five production facilities are consistent with unrecorded factors not materially affecting accuracy (Section 4.4, Table 3): realized damage stays within 8% of MATEL's predictions for both corrugate boxes and flexible bags. Associates follow the packaging recommendation for the large majority of shipments, so unrecorded judgment affects only a small share of decisions. Deviations are mostly upgrades to a corrugate box, the more protective option, which lowers realized damage in operation. Because they are rare, their influence on the estimates is limited, consistent with the agreement between predicted and realized damage.

Third, Appendix D.1 checks the validity of the observational reference. Within product × route, facility type, ship speed, region, and shipment composition all have low standardized mean differences between arms (Figure D.1), and all 45 pairwise contrasts keep their sign when route is held fixed (Figure D.2). The observed contrasts thus show no sign of major confounding.

Q2. Since the real-world evaluation uses the model’s own AIPW pseudo-outcomes, could this inflate MATEL’s performance?

In the original manuscript, all methods were scored against the same doubly robust pseudo-outcome. Following your comment, we now also score DML on a pseudo-outcome built from its own cross-fitted nuisances. DML's BLP slopes improve slightly, but the conclusions hold (Table D.1, Figure D.4; Appendix D.2), so the shared reference does not inflate MATEL's performance.

Q3. The synthetic data-generating process appears specifically tailored to favor the proposed pairwise renormalization.

Inspired by your comment, we added stress tests in Appendix E.1 that compare MATEL with matched baselines under mild and extreme settings (Table E.2). In three scenarios, every pack type is assigned from a model fitted on real data instead of our designed overlap; in two others, damage has lighter or heavier tails than our loss assumes. MATEL outperforms matched variants sharing its backbone and outcome heads when overlap narrows or tails grow heavier, and performs comparably under more uniform assignment and lighter tails (lower ATE error, slightly higher PEHE than HydraNet (adapted)). Across all five settings, its ATE error stays at 192–329 DPMO after reconciliation, with 87–93% of oracle uplift.

Q4. The theoretical novelty is incremental ...notation is dense... add engineering insights, the real-world data distribution, and the deployment pipeline.

We have added Section 4.4 to show the recent production impact of MATEL in five fulfillment facilities, our engineering insights from the deployment along with the pipeline (Appendix D.3, Figure D.9), and the prediction accuracy checks against realized damage data (Table 3). For the real-world data distribution, the original draft already includes Table D.3 and Figures D.5–D.8 in Appendix D. Additionally, we added Appendix D.1 to describe how shipments are aggregated per product, route, and pack type over a rolling one-year window refreshed at each weekly retraining, and to report the distribution of operational covariates across pack types (Figure D.1).

We understand that the dense theory and formulas in the original manuscript were distracting. Following your recommendation, we shortened the theoretical content and streamlined how it is presented. Section 3.3 now states each guarantee in words and cites only the three theorems, the reconciliation proposition, and its corollary. Its formulas and auxiliary results now sit in Appendix C.6, and the remarks were shortened or folded into the statements they qualify.

We would also like to be specific about the novelties. MATEL does build on existing components, including the Tweedie loss and TMLE targeting. Its main contributions are the renormalized pairwise propensity and targeting, whose bounds depend on relative rather than absolute overlap; the hybrid encoder, which keeps outcome-only features out of the propensity; and the transitive reconciliation.

Thank you again for your insightful comments; we hope the revision addresses them.

Reviewer_H43DOpenReview ↗

Response for 19143

Thanks to the authors for the response; my assessment is now more accurate.

Q2's fair-comparison check and Q3's stress tests can be verified directly in the revision, and Q4's promised Section 4.4, Table 3, and Figure D.9 do appear in the main text. The only remaining uncertainty is Q1: the three robustness checks are informative but none places a causal bound on unobserved confounding. I suggest adding an E-value as the primary bound, which directly answers how strong an unobserved confounder would need to be to drive the 2.87 pp gain to zero, with OVB calibration and a negative-control test as complementary checks.

Overall, I consider this a 3-point paper, and I am willing to raise my confidence to 4.

AuthorsOpenReview ↗

Follow-up on H43D's feedback

Thank you so much for your prompt and thoughtful reply over the weekend, and for raising your confidence to 4.

Following your suggestion, we added an E-value for the 2.87 pp production saving (Section 4.4 and Appendix D.2). The saving disappears only if true damage under MATEL's decisions is at least 7.7 times its estimate, which corresponds to an E-value of 14.8, so a hidden confounder would need a risk ratio of at least 14.8 with both pack choice and damage to erase the saving. As an even more conservative evaluation, assuming the shadow decisions caused no damage at all, the E-value is still 13.2. In production, realized damage has stayed between 0.97 and 1.08 times MATEL's predictions.

We are grateful for the productive rebuttal discussion and for your kind advice in improving our manuscript. We hope the revision may improve your perspective on the paper quality, and if there is any further improvement that would help, we will be happy to devote our full effort to it before the end of the discussion period. We fully respect your assessment, and iterating on your comments has made the paper substantially stronger. Thank you again for all your efforts on our paper, and we wish you the best in your endeavors!

Reviewer_Rf4POpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

Paper introducesMulti-Arm Treatment Effect Learner (MATEL), a causal deep-learning framework designed for selecting optimal retail packaging.

Contributions:

  • Novel methodology since assignment is confounded, and the bandit formulation has full-information.
  • MATEL has a shared tabular backbone and leverages tweedie heads, since the normal gaussian heads are misspecified.
  • Separate hybrid propensity encoder over confounders W only, not outcome-only X.
  • Independent sigmoids (one-vs-rest) instead of softmax to avoid simplex coupling of rare arms.
  • Transitive reconciliation, necessary since MATEL debiases each pair independently
  • Doubly-robust correction under sparsity

Method is validated on both real-world and synthetic data. Authors also produce a production simulation to test the whole ecosystem.

Critical strengths

Paper introduces a novel applications with its own specifications. It presents an elegant and robust theoretical formulation: doubly robust, bounded, hybrid-representation layers, additive pairwise reconciliation. The theoretical framework is strong. It is coupled with convincing validation, be it over synthetic or real data.

Beyond e-commerce packaging, the paper presents a solution for sparse pairwise overlap via renormalization, plus per-unit reconciliation for transitivity. The perform pairwise renormalization while clipping a debiased estimated TMLE. This is a clever formulation. Renormalized weights depend only on relative overlap, not absolute margin. With clipped fluctuation, corrections and contrasts stay uniformly bounded. The independently-debiased pairs are then mapped to one additive level vector, achieving reconciliation.

Critical weaknesses

The main critique is the lack of ground truth. Given this is a novel application, it is unclear what is the optimal way to package items. In addition, no real competitors are run against MATEL, since their implementations weren't transferable. The experiments are relative improvements over production, or per-pair benchmarks like Dragon Net, but the floor remains unknown. Given the novel application, I do not hold this against authors.

Some of the writing can be improved. Here are some nit picks

  • Line 110: introduce DPMO
  • Line 128: introduce positivity before first usage it. Explain business feasibility
  • Equation (2): To ensure user don't think "h" is part of the sum., I suggest putting parenthesis. You may want to explicitly addd in text how you accommodate items that can only be shipped in a certain package type. Now it is mostly elucidated in appendix
  • Should you introduce the training objective (L236) before the clever covariate?
Questions for authors

Clarification: why wasn't HydraNet transferable and used as a competing method?

2 comments
AuthorsOpenReview ↗

Rebuttal response to reviewer Rf4P

Thank you for your positive and encouraging review, and for your recognition of the theoretical formulation as well as the methodology itself. We greatly appreciate the time and effort you devoted to our paper. To address your points on ground-truth evaluation and comparisons, we revised the manuscript with additional evidence and clarifications.

Q1: The main critique is the lack of ground truth… the floor remains unknown. Given the novel application, I do not hold this against authors.

Thank you for your understanding. It is indeed challenging to obtain ground-truth effects that reflect real-world confounding and outcome distributions. To strengthen confidence in MATEL's performance, we added evidence from two sides.

  • Known-truth synthetic data. MATEL outperforms the matched benchmark methods, especially when treatment overlap narrows or outcome tails grow heavier (Table 2). New stress tests, which move the assignment mechanism and the outcome tail away from our original design, show the same pattern (Table E.2, Appendix E.1).

  • Real-world data. We now report a large-scale production pilot in five fulfillment facilities, chosen before launch to differ in equipment, region, packaging supply, and capacity (Section 4.4). MATEL's damage predictions hold up in live operation: realized problem-solve DPMO stays within 8% of MATEL's predictions (Figure 2, Table 3). Compared with the shadow rule-based decisions on the same shipments, MATEL cut the corrugate box share by 19.8% and delivered at least a 2.87 pp net cost improvement. This is a conservative lower bound, because the saving is credited only to bag shipments that the shadow decisions confirm would have been boxed.

We hope these analyses provide evidence of MATEL's effectiveness on data with known ground truth and of its business impact in production.

Q2: Clarification: why wasn't HydraNet transferable and used as a competing method?

HydraNet's original implementation uses squared-error outcomes, unweighted rows, and a fixed number of arms, which does not suit zero-inflated, heavy-tailed DPMO outcomes. However, the structure can be adapted to match MATEL's design as an ablation. Therefore, we now include HydraNet (adapted) in Tables 1 and 2 (configuration in Appendix A, Table A.1). It keeps HydraNet's shared representation, softmax propensity, and multi-arm targeted regularization, on MATEL's backbone and Tweedie heads. On real data it fails to reach the directional agreement with the observed reference on both sparse arms, while MATEL is correct for all arms (Table 1). On synthetic data, its ATE error grows quickly as overlap weakens (302 to 403–524 DPMO), while MATEL's stays bounded (Table 2). In the two hardest stress settings it reaches 344 and 448 DPMO, against MATEL's 192 and 329 (Table E.2).

Q3: Line 110: introduce DPMO. Line 128: introduce positivity; explain business feasibility. Equation (2): add parentheses. Introduce the training objective before the clever covariate.

Thank you for these careful suggestions. We now define DPMO at its first use (Section 2), clarify that positivity is required only for the pack types a product can feasibly ship in (Section 2.1), and add parentheses in the cost equation (Equation (1)). For the last point, the training objective uses the clever covariate in its pairwise term, so we keep the clever covariate first and added a sentence to clarify (Section 3.1).

Thank you again for your constructive comments. We sincerely hope you find the revised version satisfactory.

AuthorsOpenReview ↗

Follow-up to Reviewer Rf4P

Dear Reviewer Rf4P,

Thank you again for your positive and encouraging review, and for describing our theoretical formulation as elegant and robust. We know this is a busy period, and we truly appreciate the time you have already devoted to our paper.

We wanted to make sure our response has been delivered before the discussion period closes. Following your comments, we made three main additions. First, we now include HydraNet (adapted) as a competitor on both real and synthetic data, explaining why the original implementation does not transfer to zero-inflated, heavy-tailed damage outcomes (Tables 1 and 2, Appendix A). Second, to address the lack of ground truth concern, we enriched the ablation study on the synthetic data and report a large-scale production deployment in five fulfillment facilities, where MATEL cut the corrugate box share by 19.8% against shadow rule-based decisions on the same shipments, with realized damage within 8% of its predictions (Section 4.4, Table 3). Third, we addressed the writing suggestions on DPMO, positivity, Equation (2), and the order of the training objective.

The discussion with the other reviewers has also been productive, and we believe the revised paper is substantially stronger as a result. We are very eager to hear your thoughts on the changes and our responses to your questions and will try our best to respond before the deadline today.

Thank you again for your support of our work. Wish you all the best!

Reviewer_K8DGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MATEL, a method that estimates damage outcomes under different packaging options to guide packaging decisions. Specifically, the method combines Tweedie outcome heads, a separate propensity encoder, pairwise calibration, and a final step that ensures consistent predictions across options. The paper evaluates MATEL on observational data, synthetic experiments, and a simulation that reports lower costs and box usage.

Critical strengths
  1. The paper studies the packaging decision problem, an important practical problem where predicting observed damage alone is not sufficient.
  2. The architecture is well designed for the application, with each component serving a clear purpose.
  3. The paper provides detailed theory, implementation, and evaluation, and the comparison between shared and separate encoders supports the design choice.
Critical weaknesses
  1. The theoretical guarantees do not directly cover the product-level predictions used by the optimizer. For example, Theorem 2 concerns an average effect within a treatment pair, and the appendix acknowledges remaining outcome model bias.
  2. The real-data evaluation relies on an observational reference, which does not automatically control differences in route, facility, time, or shipment context. Therefore, the reported directional agreement and interval coverage do not establish accuracy against true causal effects.
  3. The comparisons do not fully isolate the contribution of MATEL, since the baselines differ in model capacity, clipping, and outcome loss. The paper should conduct a matched comparison with pairwise DML or TMLE under the same settings.
  4. Table 2 evaluates effects before the final consistency step, although the optimizer uses the final predictions. In addition, the simulation disables capacity constraints and does not compare other estimators under the same optimizer, so the source of the reported savings remains unclear.
Questions for authors
  1. Which theoretical guarantees apply to the final product-level predictions and their intervals?
  2. How does the observational reference control differences in route, facility, time, and shipment context?
  3. How does MATEL compare with pairwise DML or TMLE using the same models, propensity normalization, and clipping?
  4. What are the final prediction errors and decision gains when other estimators use the same optimizer?
5 comments
AuthorsOpenReview ↗

Rebuttal response to reviewer K8DG

Thank you for your careful and constructive review, and for recognizing the importance of the problem, the design, and the theory and implementation. We respond to each question below.

Q1. Which theoretical guarantees apply to the final product-level predictions and their intervals?

The optimizer consumes the reconciled per-product levels V^(z)\hat V(z) (Figure 1), and Table C.1 maps each result to the quantity it covers. Per product, the levels are bounded regardless of how small the absolute overlap is (Theorem 1) and are transitive (Proposition 2). The new Corollary 1 shows that reconciliation is a projection onto a convex set containing the true levels, so it cannot increase the combined error of its inputs. Double robustness (Theorem 2) holds at the pair level, and nominal coverage (Proposition 4) holds for aggregate intervals. None of the compared benchmarks, nor doubly robust CATE learners in general, offers a per-product guarantee as strong as the pair-level one: doubly robust learners for conditional effects attain only slower nonparametric rates, and DML and TMLE, like MATEL, give their strongest guarantees for averages. Empirically, the per-product intervals cover 89–97% of observed contrasts (Table 1), and realized production damage stays within 8% of MATEL's predictions (Table 3).

Q2. How does the observational reference control differences in route, facility, time, and shipment context?

Appendix D.1 now documents the reference. Contrasts are formed only within product × route cells in which both pack types shipped, so product attributes, and with them the rule-based eligibility behind most pack decisions, as well as route, are fixed by construction. Dropping the route stratification leaves all 45 signs unchanged (Figure D.2). The validation set is also stratified by facility, customer region, and delivery speed, and facility type, ship speed, region, and shipment composition all have standardized mean differences below 0.25 (Figure D.1). We deliberately do not stratify by time: eligibility is set per product for the whole year, so the optimizer needs the season-averaged effect that a rolling one-year reference measures. MATEL also adjusts for route, facility type, and ship speed through W. In the five-facility production pilot (Section 4.4), realized damage stays within 8% of MATEL's predictions (Table 3), which we would not expect if facility- or time-specific confounding were large.

Q3. How does MATEL compare with pairwise DML or TMLE using the same models, propensity normalization, and clipping?

Table 1 adds three matched comparisons, each changing one component of MATEL (Table A.1). MATEL (No-Renorm) replaces only the renormalized pairwise propensity and targeting with a softmax over raw gkg_k: its Box-C propensity spreads to (0.10, 0.61), the fluctuation rises from 74.3 to 102.5, and Box-C flips sign. HydraNet (adapted) keeps MATEL's backbone and Tweedie heads with a shared softmax propensity, and DML (ResNet) is pairwise DML with ResNet-PLE nuisances, the renormalized ek,be_{k,b}, and the same clip. These two match the observed direction on only 3/5 contrasts, against MATEL's 5/5, and have lower per-product agreement on every arm. On synthetic data, removing the pairwise TMLE doubles MATEL's ATE error under severe overlap (736 vs. 353 DPMO, Table 2). Appendix E.1 summarizes the components: the backbone and Tweedie heads drive ranking, pairwise targeting drives accuracy under thin overlap and heavy tails, and reconciliation makes the estimates usable by the optimizer.

Q4. What are the final prediction errors and decision gains when other estimators use the same optimizer?

After reconciliation, the form the optimizer consumes, MATEL keeps 90–92% of oracle uplift, its PEHE\sqrt{\text{PEHE}} changes by less than 4%, and its ATE error remains the lowest under weak and severe overlap (Table 2). We selected the estimator to deploy from these offline comparisons. HydraNet (adapted) and DML (ResNet) match the observed direction on only 3/5 contrasts (Table 1), the best alternatives capture at most 88–89% of oracle uplift against MATEL's 91–92% (Table 2), and several produce outputs the optimizer cannot use directly, such as negative damage or intransitive contrasts (Table E.3). We therefore deployed only MATEL and measured it against the system it replaces. Section 4.3 now explains that w=1w{=}1, which prices all measurable damage costs, is the production minimum; over w≥1w\ge1, the simulated saving is 1.94–3.74 pp (Table D.2). Under actual capacity constraints, the production pilot cut the corrugate box share by 19.8% and delivered at least a 2.87 pp net cost improvement. This is a conservative lower bound, credited only to bag shipments that the shadow decisions would have boxed. Realized problem-solve DPMO stayed within 8% of MATEL's predictions (Figure 2, Table 3).

Thank you again for your constructive comments. We hope the revision addresses your concerns.

Thank you for the detailed clarifications and additional results. The revisions have largely addressed my concerns. I have one further clarification question: could you elaborate on the rationale for not stratifying by time? Packaging choices and damage risks may both vary seasonally, so adjusting for time may still be helpful even when targeting a year-averaged effect. I would appreciate your perspective on this point.

AuthorsOpenReview ↗

Follow-up to reviewer K8DG

Thank you so much for the follow-up and for this inspiring comment. We are very glad that the revisions have addressed most of your concerns, and we truly appreciate the care you have put into our paper.

  • You raise an excellent point: controlling for time is the ideal design, and a time-aware model could further improve the granularity of MATEL's estimates. In the current version, two constraints led us to target a time-invariant effect. First, data sparsity: both training and evaluation rely on products shipped in two pack types on the same route, and on the sparse arms such products are few. Splitting them by month or quarter would push most sparse-arm contrasts below the minimum shipments our observational reference requires, so time-specific effects could not be validated on real data. Second, from a business perspective, time-specific estimates would change recommendations from period to period, which would require upstream procurement and downstream facility operations to plan and absorb monthly shifts in the pack mix. We therefore view time-aware estimation as an extension of both the model and the surrounding systems, rather than a change we can make reliably within this revision.

  • Within the current design, the year-averaged effect is the quantity the optimizer needs, since decisions are made per product rather than per period, and a time-stratified model would in any case be averaged back to the product level. Stratifying by time would change this averaged effect only if a product's two pack types were used in systematically different parts of the year while damage also varied across those parts. This is limited here, because pack decisions follow rule-based recommendations driven by product attributes that do not change with the season, and the within-product comparison holds those attributes fixed. Seasonal variation mostly appears as a co-movement: pack-type availability differs across facilities and regions, which enter WW as confounders, and when more protective pack types are available, aggregate damage can look lower, driven by a shift in the network's mix. Aggregating over a full annual cycle also means every season is represented, so the estimated effect is not tilted toward any particular part of the year. MATEL thus provides a static policy that balances seasonal variation in performance, which is the granularity the surrounding systems can operate at until they support finer, time-varying decisions.

  • The current design also matches how the estimates are used and tracks change over time in operation. MATEL makes its recommendations in real time, one shipment at a time, but each reflects the expected best pack type for that product, so together they move the network's pack mix steadily toward a stable target that supply planning can follow. Network damage follows a gradual trend, shifting with new pack types, operational improvements, and route changes that enter MATEL through its pack-type arms and covariates, and weekly retraining on the rolling window adds the latest shipments and damage records so the estimates follow that trend. In the five-facility deployment, the predictions remain calibrated on recent shipments, with realized damage 0.97–1.08 times MATEL's predictions.

We have added this discussion to the revision (Appendix D.1) and list time-aware estimation as future work. We hope this clarifies our rationales and addresses your remaining concerns, and we look forward to your updated assessment. Thank you again for your careful reading and insightful comments throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.