Lightnews — Scholar-powered news

The Unjournal (Unjournal.org) @unjournal.bsky.social · 8h

#development #ClimateAdaptation #agricultureRetry

This paper was suggested to us by www.aiddata.org/ (by 1 author) as being among their highest-value work meriting further evaluation. (Thanks!)

See our ‘evaluating research from impact-oriented organizations’ initiative -- bit.ly/4oahxEc.

AidData — A Research Lab at William & Mary

We equip policymakers and practitioners with better evidence to improve how sustainable development investments are targeted, monitored, and evaluated. We use rigorous methods, cutting-edge tools, an...

www.aiddata.org

The Unjournal (Unjournal.org) @unjournal.bsky.social · 8h

Eva, managers also flagged identification concerns:

- Potential migration/sorting bias (healthier people moving TO irrigated areas?)

- Program timing may be endogenous (placed where conflict already decreasing?)

- Pre-trends need tighter verification (evidence of boundedly small differences)

1 2

The Unjournal (Unjournal.org) @unjournal.bsky.social · 8h

E1's suggestions:

- Update to modern TWFE methodology

- Quantify output effects "in an area with a larger buffer around the irrigation system"

- Additional robustness checks

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 8h

Key concerns from E2:

"Need more context-specific information on the causal pathway" for better investment & program design

Measurement issues: surveys used _recall_ on caloric intake rather than standard nutrition indicators

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 8h

Evaluators liked the:

- Novel combination of spatial + survey data

- Basic empirical patterns (output & nutrition) considered sound

- Important policy question

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 8h

The evaluations were split:

E1: 75/100 - Strong confidence in agricultural output increases, appreciated the methodology

E2: 46/100 - Found nutrition results "credible" but had substantial doubts about causal interpretation

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 9h

Does "Irrigation strengthen climate resilience"? -- Unjournal evaluation

https://bit.ly/3J1iMqk

BenYishay et al '24: River-based irrigation boosted #agricultural output & child #nutrition & reduced conflict in supported communities, but had negative spillovers on surrounding areas.
--->

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 9h

Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys"

Evaluation Summary and Metrics: "Irrigation Strengthens Climate Resilience: Long-term Evidence from Mali using Satellites and Surveys" for The Unjournal.

bit.ly

The Unjournal (Unjournal.org) @unjournal.bsky.social · 5d

@scipost.bsky.social congratulations on your progress & thanks for all you do.

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 5d

I love what #SciPost does & their narrative.
Much agrees w/ Unjournal.org practices.

Some differences (Us/Them)
1. Quant. ratings vs accept/reject
2. Publish all evaluations vs. only if ‘accepted’
3. Fields & applied focus
4. We pay evaluators
5. We don’t 'publish' papers, just evaluations

1 1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 7d

But having a "published object" in the ecosystem linked both to our evaluations and ratings and linked to the author's name could have value in terms of making our evaluations and ratings count career-wise.

The Unjournal (Unjournal.org) @unjournal.bsky.social · 7d

At The Unjournal we don't do the "publish" part of PRC -- we don't host the research, we just link it, & host the evaluations ,authors' responses, etc. (see unjournal.pubpub.org)

-->

The Unjournal

The Unjournal PubPub community: journal-independent evaluation of impactful work in economics, policy, and social science

unjournal.pubpub.org

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 7d

You also raise an important question:

1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 7d

Each of our evaluations cites the original paper (as well as other papers).

But our evaluations do not tend to show up as "papers citing" the paper evaluated in Gscholar.

This limits a key path to visibility, which we have been struggling with.

The Unjournal (Unjournal.org) @unjournal.bsky.social · 7d

We also periodically add these to a server feeding repec.org.

This gets most of our content into Google Scholar, which is the tool most people seem to use.

But not always in an ordered way, and there's one main thing that we're missing:

RePEc: Research Papers in Economics

RePEc is a central index of economics research, including working papers, articles and software code

repec.org

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 7d

Looking forward to reading this. At unjournal.org, we use PubPub to mint a DOI for each of the evaluations of the papers that are evaluated, as well as the evaluation summary and authors' responses.

Unjournal | Transparent Evaluation of Impactful Research

The Unjournal enables open, expert evaluation of economics and social science research, without the constraints of traditional journals. Discover how we make rigorous research more transparent.

unjournal.org

1 1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 9d

Thanks to @benbalmford.bsky.social Dr. Ben Balmford (www.benbalmford.com, LEEP, Exeter) for your strong work in this area, as well as to many diligent managers and evaluators.

www.exeter.ac.uk

The Unjournal (Unjournal.org) @unjournal.bsky.social · 9d

Climate change, environmental econ., biodiversity: Of the 50+ papers Unjournal.org commissioned for evaluation and assessment, ten focus on these areas, offering insight into methods & policy.

See our 'collection': https://bit.ly/4pKF0NT -->

Overall ratings by research/outcome area

Ratings for environment-related research

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

Sorry, the top-4, funding 4 papers each, included
@j-pal.bsky.social

(IPA funded one paper we evaluated)

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

2. To share info on which orgs tend to fund research judged to have the most potential for impact by The Unjournal team.

To help researchers & managers decide which institutions to work at, where to seek support, etc.

@80000hours.bsky.social @founderspledge.com ...

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

We provide a faster & clearer feedback loop relative to traditional academic journals

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

Why did we want to know this?

1. To make sure these organizations can benefit from the evaluations.
- Learn the results of their funding.
- Is the research seen as credible & promising by experts in that specific area?
- Should it be used to set policy and make recommendations?

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

Again, see tinyurl.com/22k6ashr for the full dashboard, including many organizations not on Bluesky.

tinyurl.com

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

University & research centers:
@research-fri.bsky.social
@yaleegc.bsky.social
@nyuad-oaa.bsky.social
@fuqua.duke.edu

1

The Unjournal (Unjournal.org) @unjournal.bsky.social · 11d

Government & international bodies:

@nasaearthwatch.bsky.social
@agencerecherche.bsky.social
@erc.europa.eu

1