Subscribe to R-sig-geo feed
This is an archive for R-sig-geo, not a forum. It is not possible to post through Nabble - you may not start a new thread nor follow up an existing thread. If you wish to post, but are not subscribed to the list through the list homepage, subscribe first (through the list homepage, not via Nabble) and post from your subscribed email address. Until 2015-06-20, subscribers could post through Nabble, but policy has been changed as too many non-subscribers misunderstood the interface.
Updated: 2 hours 20 min ago

Re: rgdal release candidate 1.5-9 rev. 1000 ready for testing

9 hours 36 min ago
Hi All:

Is there a source for binaries of PROJ >= 6 and GDAL > 3  for the Mac, other than the Anaconda ones which cause problems because they use @rpart?

Thanks,

-Roy


> On Jun 5, 2020, at 4:29 AM, Roger Bivand <[hidden email]> wrote:
>
> The release candidate of rgdal_1.5-9 is ready for testing on R-forge:
>
> https://r-forge.r-project.org/R/?group_id=884
>
> Those insisting on installing on PROJ >= 6 and GDAL < 3 must use configure argument --with-proj_api="proj_api.h"; with this used, this version builds with --no-build-vignettes and installs and checks OK on PROJ 7.0.1 and GDAL 2.2.4 with --with-proj_api="proj_api.h".
>
> Otherwise checked OK with PROJ 4.8.0, 4.9.2, 4.9.3 and 5.2.0 with GDAL 1.11.4; with PROJ 5.2.0 and GDAL 2.2.4, 2.3.2 and 2.4.2; with PROJ 6.3.2 and GDAL 3.0.4; with PROJ 7.0.1 and GDAL 3.0.4 and 3.1.0.
>
> All who have indicated issues with source installs are asked to try the release candidate and to report back here by midnight CEST Monday 8 June. If no indications are forthcoming, I'll assume that problems with 1.5-8 are resolved.
>
> Roger
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
**********************
"The contents of this message do not reflect any position of the U.S. Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: [hidden email] www: https://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

rgdal release candidate 1.5-9 rev. 1000 ready for testing

12 hours 1 min ago
The release candidate of rgdal_1.5-9 is ready for testing on R-forge:

https://r-forge.r-project.org/R/?group_id=884

Those insisting on installing on PROJ >= 6 and GDAL < 3 must use configure
argument --with-proj_api="proj_api.h"; with this used, this version builds
with --no-build-vignettes and installs and checks OK on PROJ 7.0.1 and
GDAL 2.2.4 with --with-proj_api="proj_api.h".

Otherwise checked OK with PROJ 4.8.0, 4.9.2, 4.9.3 and 5.2.0 with GDAL
1.11.4; with PROJ 5.2.0 and GDAL 2.2.4, 2.3.2 and 2.4.2; with PROJ 6.3.2
and GDAL 3.0.4; with PROJ 7.0.1 and GDAL 3.0.4 and 3.1.0.

All who have indicated issues with source installs are asked to try the
release candidate and to report back here by midnight CEST Monday 8 June.
If no indications are forthcoming, I'll assume that problems with 1.5-8
are resolved.

Roger

--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway

Re: How to efficiently generate data of neighboring points

13 hours 2 min ago
Thank you once again. To clarify, which is more suitable, end of year water
levels or yearly average measure of water levels?

Also below are a few more notes to throw more light on my variables/data:

These wells are solely for irrigation purposes and are
irrigator/farmer-owned and operated.
No farmer/irrigator moves to another well not owned by him. The only reason
to suspect any spatial externalities is because the wells share a common
aquifer.
And this is essentially what I am testing.

It is also understood that there are not much variation in the geography
and geology of the study region.

I have data a number of well specific features in addition to the water
level. I also have some farm data including cropping and technology use
data. No soil data though.
No recharge data too as well.

In fact, I agree a lot factors can come to play here and I may not have or
observe all but I was thinking I could incorporate some fixed effects
to take care of those, especially for those I suspect (or perhaps by
theory) are likely to not vary much in terms of their effect on
irrigation(pumping) decisions across farmers
or effect on water level.

My panel is rather a short one: I have a five year panel data.

Given the above, is it still not advisable to use any spatial econometric
analysis? Just a simple OLS will suffice?

Thanks.
----------------------
Lom





On Fri, Jun 5, 2020 at 3:51 AM Roger Bivand <[hidden email]> wrote:

> On Fri, 5 Jun 2020, Lom Navanyo wrote:
>
> > I fully agree with you and appreciate the listed benefits of not taking
> > things private. I was just trying to be sure the forum here is
> appropriate
> > and receptive of a beginner like me.
> >
> > To be more explicit with regards to my observations, y is amount of
> > water withdrawal from wells and an important variable in x is (height
> > of) water level in the wells. These are end of year figures. I am using
> > the aggregations (sum for y and mean for water level) by band as spatial
> > neighborhood variables. There will be one or two indicator variables
> > also in x. I hope these do not present additional hurdles.
>
> There are several further questions. If water level is measured at
> end-of-year, it is instantaneous at that point, and will depend on level a
> year earlier plus inflow from the movements of the water table
> (precipitation, soils and surface geology, maybe geology if deeper wells),
> minus evaporation (if an open well) and extraction. However, your y
> (extraction) is probably measured over an interval (1 Jan - 31 Dec?). It
> does not depend on level unless level is 0, but depends on the closeness
> of people extracting water for domestic, agricultural or other use.
>
> All else equal, you would expect changes in the level in a well to depend
> on inputs, evaporation and extraction, and extraction at that well and
> other nearby wells (which may experience falls in the ground water table
> level not because the water was extracted from those wells, but at
> neighbouring wells. You may also see users shifting to neerby wells if
> their closest well runs dry.
>
> So you probably need to start with a deterministic hydrological model, and
> you need much more information about who extracts and why. Say in India,
> you would also need price data - apparently free water has led to
> over-extraction.
>
> So I would advise against any spatial econometric analysis of the data you
> have, because so much is going on in the system as a whole that you cannot
> control if all the data you have is as you describe. I also understand
> better why well water level is endogeneous, but am sure that IV will not
> help, since the level is being driven partly by a deterministic
> hydrological system which differs from well to well, and extraction varies
> by demand.
>
> Has anyone worked with this kind of data? Any ideas or contributions more
> helpful than the above?
>
> Roger
>
> >
> > I am thinking Proximity is relevant in testing spatial
> > dependency/externality.
> >
> > I will consider splm package  and the SLX model.
> >
> > Thank you.
> > ---------------
> > Lom
> >
> > On Thu, Jun 4, 2020 at 2:52 PM Roger Bivand <[hidden email]> wrote:
> >
> >> On Thu, 4 Jun 2020, Lom Navanyo wrote:
> >>
> >>> Thank you. Yes, the OLS is biased and my plan is to use a 2SLS
> approach.
> >> I
> >>> have a variable I intend to use as an IV for y.
> >>> I have seen a few papers use this approach. Will this approach not
> >> correct
> >>> for the endogeneity?
> >>>
> >>> Actually, I am not sure if this is a right forum or perhaps if it's
> >>> appropriate or acceptable to you to take this one-on-one with you for
> >> help:
> >>
> >> I do not offer private help. That would presuppose that one person has
> the
> >> answer. It would also presuppose that all exchanges are only read by the
> >> original poster and direct participants, while in fact others may join
> in,
> >> or follow a thread, or find the thread by searching: google supports the
> >> list:r-sig-geo search tag. If the thread goes private, that search is
> >> fruitless.
> >>
> >>> My model actually looks like this: y= f(y, x)  + e.
> >>> Aside the endogeneity of y (which I intend to instrument by another
> >>> variable z), there is simultaneity between y and x.
> >>> I intend to use the lag of x as instrument for x.  Given that I am
> >> seeking
> >>> to test spatial dependency, do you see some fatal flaws with my
> approach?
> >>>
> >>
> >> What is the support of your observations, point, or are they
> aggregations?
> >> Why may proximity make a difference - often, apparent spatial
> >> autocorrelation is caused by observing inappropriate entities, or by
> >> omitting covariates, or by using the wrong functional form.
> >>
> >>
> >>> I have also seen other empirical approaches like static and dynamic
> >> spatial
> >>> panel data modelling. I will be reviewing them also to see suitability
> >> for
> >>> my objective.
> >>> But, any further directions or suggestions are highly appreciated.
> >>
> >> If the data are spatial panel, you can look at the splm package.
> >> Personally, I have never found instruments any use at all, because the
> >> instruments are typically at best weak because of shared spatial
> processes
> >> with the response, unless the model is really well specified from known
> >> theory. In space, almost everything is close to endogeneous unless the
> >> opposite is demonstrated. So causal relationships are less worthwhile,
> >> because they are at best conditional on omitted variables and
> >> autocorrelation engendered by the choice of observational entities.
> >>
> >> Further, because spatial processes are driven by the inverse matrix of
> the
> >> input graph of proximate neighbours (the covariance matrix of the
> spatial
> >> process), you don't need to start from more than the first order
> >> neighbours. Maybe your x has the same spatial pattern as y, so that the
> >> residuals are white noise with no spatial structure.
> >>
> >> Recently, analysts prefer to start with the SLX model (Halleck Vega &
> >> Elhorst 2015 and others), so that might be worth exploring. If only the
> >> direct impacts seem important, OLS may be enough.
> >>
> >> Hope this helps,
> >>
> >> Roger
> >>
> >>>
> >>> Thanks,
> >>> -------------------
> >>> Lom
> >>>
> >>>
> >>>
> >>> On Thu, Jun 4, 2020 at 3:48 AM Roger Bivand <[hidden email]>
> wrote:
> >>>
> >>>> On Thu, 4 Jun 2020, Lom Navanyo wrote:
> >>>>
> >>>>> Thank you very much for your support. This gives me what I need and I
> >>>> must
> >>>>> say listw2sn() is really great.
> >>>>>
> >>>>> Why do I need the data in the format as in dataout? I am trying to
> test
> >>>>> spatial dependence (or neighborhood effect) by running a regression
> >>>>> model that entails pop_size_it = beta_1*sum of pop_size of point i's
> >>>>> neighbors within a specified radius. So my plan is to get the
> neighbors
> >>>>> for each focal point as per the specified bands and their attributes
> >> (eg
> >>>>> pop_size) so I can can add them (attribute) by the bands.
> >>>>
> >>>> Thanks, clarifies a good deal. Maybe look at the original localG
> >> articles
> >>>> for exploring distance relationships (Getis and Ord looked at
> HIV/AIDS);
> >>>> ?spdep::localG or
> >> https://r-spatial.github.io/spdep/reference/localG.html.
> >>>>
> >>>> Further note at OLS is biased as you have y = f(y) + e, so y on both
> >>>> sides. The nearest equivalent for a single band is
> >> spatialreg::lagsarlm()
> >>>> with listw=nb2listw(wd1, style="B") to get the neighbour sums through
> >> the
> >>>> weights matrix. So both your betas and their standard errors are
> >> unusable,
> >>>> I'm afraid. You are actually very much closer to ordinary kriging,
> >> looking
> >>>> at the way in which distance attenuates the correlation in value of
> >>>> proximate observations.
> >>>>
> >>>> Hope this clarifies,
> >>>>
> >>>> Roger
> >>>>
> >>>>>
> >>>>> I am totally new to the area of spatial econometrics, so I am taking
> >>>> things
> >>>>> one step at a time. Some readings suggest I may need distance matrix
> or
> >>>>> weight matrix but for now I think I should try the current approach.
> >>>>>
> >>>>> Thank you.
> >>>>>
> >>>>> -------------
> >>>>> Lom
> >>>>>
> >>>>> On Wed, Jun 3, 2020 at 8:18 AM Roger Bivand <[hidden email]>
> >> wrote:
> >>>>>
> >>>>>> On Wed, 3 Jun 2020, Lom Navanyo wrote:
> >>>>>>
> >>>>>>> I had the errors with rtree using R 3.6.3. I have since changed to
> R
> >>>>>> 4.0.0
> >>>>>>> but I got the same error.
> >>>>>>>
> >>>>>>> And  yes, for Roger's example, I have the objects wd1, ... wd4, all
> >>>> with
> >>>>>>> length 101. I think my difficulty is my inability to output the
> list
> >>>>>>> detailing the point IDs t50_fid.
> >>>>>>
> >>>>>> library(spData)
> >>>>>> library(sf)
> >>>>>> projdata<-st_transform(nz_height, 32759)
> >>>>>> pts <- st_coordinates(projdata)
> >>>>>> library(spdep)
> >>>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> >>>>>> bds <- c(0, bufferR)
> >>>>>> wd1 <- dnearneigh(pts, bds[1], bds[2])
> >>>>>> wd2 <- dnearneigh(pts, bds[2], bds[3])
> >>>>>> wd3 <- dnearneigh(pts, bds[3], bds[4])
> >>>>>> wd4 <- dnearneigh(pts, bds[4], bds[5])
> >>>>>> sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
> >>>>>> sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
> >>>>>> sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
> >>>>>> sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
> >>>>>> sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
> >>>>>> sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
> >>>>>> sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
> >>>>>> sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
> >>>>>> data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3,
> >>>> sn_band4))
> >>>>>> class(data_out) <- "data.frame"
> >>>>>> table(data_out$band)
> >>>>>> data_out$ID_from <- projdata$t50_fid[data_out$from]
> >>>>>> data_out$ID_to <- projdata$t50_fid[data_out$to]
> >>>>>> data_out$elev_from <- projdata$elevation[data_out$from]
> >>>>>> data_out$elev_to <- projdata$elevation[data_out$to]
> >>>>>> str(data_out)
> >>>>>>
> >>>>>> The "spatial.neighbour" representation was that used in the S-Plus
> >>>>>> SpatialStats module, with "from" and "to" columns, and here drops
> >>>>>> no-neighbour cases gracefully. So listw2sn() comes in useful
> >>>>>> for creating the output, and from there, just look-up in the
> >>>>>> input data.frame. Observations here cannot be their own neighbours.
> >>>>>>
> >>>>>> It would be relevant to know why you need these, are you looking at
> >>>>>> variogram clouds?
> >>>>>>
> >>>>>> Hope this clarifies,
> >>>>>>
> >>>>>> Roger
> >>>>>>
> >>>>>>>
> >>>>>>> ---------
> >>>>>>> Lom
> >>>>>>>
> >>>>>>> On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Roger's example works for me and gives a list of length 101. I did
> >>>> have
> >>>>>>>> some issues that were resolved by updating packages. I'm using R
> >> 3.6.3
> >>>>>> on
> >>>>>>>> macOS 10.15.4. I also use rtree successfully on Windows 10 with R
> >>>> 3.6.3.
> >>>>>>>>
> >>>>>>>> Kent
> >>>>>>>>
> >>>>>>>> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]
> >
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>>>>>>>
> >>>>>>>>>> rtree uses Euclidean distance so the points should be in a
> >>>> coordinate
> >>>>>>>>>> system where this makes sense at least as a reasonable
> >>>> approximation.
> >>>>>>>>>
> >>>>>>>>> I tried the original example:
> >>>>>>>>>
> >>>>>>>>> remotes::install_github("hunzikp/rtree")
> >>>>>>>>> library(spData)
> >>>>>>>>> library(sf)
> >>>>>>>>> projdata<-st_transform(nz_height, 32759)
> >>>>>>>>> library(rtree)
> >>>>>>>>> pts <- st_coordinates(projdata)
> >>>>>>>>> rt <- RTree(st_coordinates(projdata))
> >>>>>>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> >>>>>>>>> wd1 <- withinDistance(rt, pts, bufferR[1])
> >>>>>>>>>
> >>>>>>>>> but unfortunately failed (maybe newer Boost headers than yours?):
> >>>>>>>>>
> >>>>>>>>> Error in UseMethod("withinDistance", rTree) :
> >>>>>>>>>    no applicable method for 'withinDistance' applied to an object
> >> of
> >>>>>>>>> class
> >>>>>>>>> "c('list', 'RTree')"
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Kent
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <
> [hidden email]>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
> >>>>>>>>>>>>> From: Lom Navanyo <[hidden email]>
> >>>>>>>>>>>>> To: [hidden email]
> >>>>>>>>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of
> >>>>>> neighboring
> >>>>>>>>>>>>>         points within specified radii (distances) for each
> >> point
> >>>>>> in a
> >>>>>>>>>>> given
> >>>>>>>>>>>>>         points data set.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hello,
> >>>>>>>>>>>>> I have data set of about 3400 location points with which I am
> >>>>>> trying
> >>>>>>>>> to
> >>>>>>>>>>>>> generate data of each point and their neighbors within
> defined
> >>>>>> radii
> >>>>>>>>>>> (eg,
> >>>>>>>>>>>>> 0.25, 1, and 3 miles).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> The rtree package is very fast and memory-efficient for
> >>>>>>>>> within-distance
> >>>>>>>>>>>> calculations.
> >>>>>>>>>>>> https://github.com/hunzikp/rtree
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks! Does this also apply when the input points are in
> >>>>>> geographical
> >>>>>>>>>>> coordinates?
> >>>>>>>>>>>
> >>>>>>>>>>> Roger
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Kent Johnson
> >>>>>>>>>>>> Cambridge, MA
> >>>>>>>>>>>>
> >>>>>>>>>>>>       [[alternative HTML version deleted]]
> >>>>>>>>>>>>
> >>>>>>>>>>>> _______________________________________________
> >>>>>>>>>>>> R-sig-Geo mailing list
> >>>>>>>>>>>> [hidden email]
> >>>>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Roger Bivand
> >>>>>>>>>>> Department of Economics, Norwegian School of Economics,
> >>>>>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>>>>>>>> https://orcid.org/0000-0003-2392-6140
> >>>>>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Roger Bivand
> >>>>>>>>> Department of Economics, Norwegian School of Economics,
> >>>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>>>>>> https://orcid.org/0000-0003-2392-6140
> >>>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Roger Bivand
> >>>>>> Department of Economics, Norwegian School of Economics,
> >>>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>>> https://orcid.org/0000-0003-2392-6140
> >>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> Roger Bivand
> >>>> Department of Economics, Norwegian School of Economics,
> >>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>> https://orcid.org/0000-0003-2392-6140
> >>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>
> >>>
> >>
> >> --
> >> Roger Bivand
> >> Department of Economics, Norwegian School of Economics,
> >> Helleveien 30, N-5045 Bergen, Norway.
> >> voice: +47 55 95 93 55; e-mail: [hidden email]
> >> https://orcid.org/0000-0003-2392-6140
> >> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: How to efficiently generate data of neighboring points

14 hours 39 min ago
On Fri, 5 Jun 2020, Lom Navanyo wrote:

> I fully agree with you and appreciate the listed benefits of not taking
> things private. I was just trying to be sure the forum here is appropriate
> and receptive of a beginner like me.
>
> To be more explicit with regards to my observations, y is amount of
> water withdrawal from wells and an important variable in x is (height
> of) water level in the wells. These are end of year figures. I am using
> the aggregations (sum for y and mean for water level) by band as spatial
> neighborhood variables. There will be one or two indicator variables
> also in x. I hope these do not present additional hurdles.
There are several further questions. If water level is measured at
end-of-year, it is instantaneous at that point, and will depend on level a
year earlier plus inflow from the movements of the water table
(precipitation, soils and surface geology, maybe geology if deeper wells),
minus evaporation (if an open well) and extraction. However, your y
(extraction) is probably measured over an interval (1 Jan - 31 Dec?). It
does not depend on level unless level is 0, but depends on the closeness
of people extracting water for domestic, agricultural or other use.

All else equal, you would expect changes in the level in a well to depend
on inputs, evaporation and extraction, and extraction at that well and
other nearby wells (which may experience falls in the ground water table
level not because the water was extracted from those wells, but at
neighbouring wells. You may also see users shifting to neerby wells if
their closest well runs dry.

So you probably need to start with a deterministic hydrological model, and
you need much more information about who extracts and why. Say in India,
you would also need price data - apparently free water has led to
over-extraction.

So I would advise against any spatial econometric analysis of the data you
have, because so much is going on in the system as a whole that you cannot
control if all the data you have is as you describe. I also understand
better why well water level is endogeneous, but am sure that IV will not
help, since the level is being driven partly by a deterministic
hydrological system which differs from well to well, and extraction varies
by demand.

Has anyone worked with this kind of data? Any ideas or contributions more
helpful than the above?

Roger

>
> I am thinking Proximity is relevant in testing spatial
> dependency/externality.
>
> I will consider splm package  and the SLX model.
>
> Thank you.
> ---------------
> Lom
>
> On Thu, Jun 4, 2020 at 2:52 PM Roger Bivand <[hidden email]> wrote:
>
>> On Thu, 4 Jun 2020, Lom Navanyo wrote:
>>
>>> Thank you. Yes, the OLS is biased and my plan is to use a 2SLS approach.
>> I
>>> have a variable I intend to use as an IV for y.
>>> I have seen a few papers use this approach. Will this approach not
>> correct
>>> for the endogeneity?
>>>
>>> Actually, I am not sure if this is a right forum or perhaps if it's
>>> appropriate or acceptable to you to take this one-on-one with you for
>> help:
>>
>> I do not offer private help. That would presuppose that one person has the
>> answer. It would also presuppose that all exchanges are only read by the
>> original poster and direct participants, while in fact others may join in,
>> or follow a thread, or find the thread by searching: google supports the
>> list:r-sig-geo search tag. If the thread goes private, that search is
>> fruitless.
>>
>>> My model actually looks like this: y= f(y, x)  + e.
>>> Aside the endogeneity of y (which I intend to instrument by another
>>> variable z), there is simultaneity between y and x.
>>> I intend to use the lag of x as instrument for x.  Given that I am
>> seeking
>>> to test spatial dependency, do you see some fatal flaws with my approach?
>>>
>>
>> What is the support of your observations, point, or are they aggregations?
>> Why may proximity make a difference - often, apparent spatial
>> autocorrelation is caused by observing inappropriate entities, or by
>> omitting covariates, or by using the wrong functional form.
>>
>>
>>> I have also seen other empirical approaches like static and dynamic
>> spatial
>>> panel data modelling. I will be reviewing them also to see suitability
>> for
>>> my objective.
>>> But, any further directions or suggestions are highly appreciated.
>>
>> If the data are spatial panel, you can look at the splm package.
>> Personally, I have never found instruments any use at all, because the
>> instruments are typically at best weak because of shared spatial processes
>> with the response, unless the model is really well specified from known
>> theory. In space, almost everything is close to endogeneous unless the
>> opposite is demonstrated. So causal relationships are less worthwhile,
>> because they are at best conditional on omitted variables and
>> autocorrelation engendered by the choice of observational entities.
>>
>> Further, because spatial processes are driven by the inverse matrix of the
>> input graph of proximate neighbours (the covariance matrix of the spatial
>> process), you don't need to start from more than the first order
>> neighbours. Maybe your x has the same spatial pattern as y, so that the
>> residuals are white noise with no spatial structure.
>>
>> Recently, analysts prefer to start with the SLX model (Halleck Vega &
>> Elhorst 2015 and others), so that might be worth exploring. If only the
>> direct impacts seem important, OLS may be enough.
>>
>> Hope this helps,
>>
>> Roger
>>
>>>
>>> Thanks,
>>> -------------------
>>> Lom
>>>
>>>
>>>
>>> On Thu, Jun 4, 2020 at 3:48 AM Roger Bivand <[hidden email]> wrote:
>>>
>>>> On Thu, 4 Jun 2020, Lom Navanyo wrote:
>>>>
>>>>> Thank you very much for your support. This gives me what I need and I
>>>> must
>>>>> say listw2sn() is really great.
>>>>>
>>>>> Why do I need the data in the format as in dataout? I am trying to test
>>>>> spatial dependence (or neighborhood effect) by running a regression
>>>>> model that entails pop_size_it = beta_1*sum of pop_size of point i's
>>>>> neighbors within a specified radius. So my plan is to get the neighbors
>>>>> for each focal point as per the specified bands and their attributes
>> (eg
>>>>> pop_size) so I can can add them (attribute) by the bands.
>>>>
>>>> Thanks, clarifies a good deal. Maybe look at the original localG
>> articles
>>>> for exploring distance relationships (Getis and Ord looked at HIV/AIDS);
>>>> ?spdep::localG or
>> https://r-spatial.github.io/spdep/reference/localG.html.
>>>>
>>>> Further note at OLS is biased as you have y = f(y) + e, so y on both
>>>> sides. The nearest equivalent for a single band is
>> spatialreg::lagsarlm()
>>>> with listw=nb2listw(wd1, style="B") to get the neighbour sums through
>> the
>>>> weights matrix. So both your betas and their standard errors are
>> unusable,
>>>> I'm afraid. You are actually very much closer to ordinary kriging,
>> looking
>>>> at the way in which distance attenuates the correlation in value of
>>>> proximate observations.
>>>>
>>>> Hope this clarifies,
>>>>
>>>> Roger
>>>>
>>>>>
>>>>> I am totally new to the area of spatial econometrics, so I am taking
>>>> things
>>>>> one step at a time. Some readings suggest I may need distance matrix or
>>>>> weight matrix but for now I think I should try the current approach.
>>>>>
>>>>> Thank you.
>>>>>
>>>>> -------------
>>>>> Lom
>>>>>
>>>>> On Wed, Jun 3, 2020 at 8:18 AM Roger Bivand <[hidden email]>
>> wrote:
>>>>>
>>>>>> On Wed, 3 Jun 2020, Lom Navanyo wrote:
>>>>>>
>>>>>>> I had the errors with rtree using R 3.6.3. I have since changed to R
>>>>>> 4.0.0
>>>>>>> but I got the same error.
>>>>>>>
>>>>>>> And  yes, for Roger's example, I have the objects wd1, ... wd4, all
>>>> with
>>>>>>> length 101. I think my difficulty is my inability to output the list
>>>>>>> detailing the point IDs t50_fid.
>>>>>>
>>>>>> library(spData)
>>>>>> library(sf)
>>>>>> projdata<-st_transform(nz_height, 32759)
>>>>>> pts <- st_coordinates(projdata)
>>>>>> library(spdep)
>>>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>>>>>> bds <- c(0, bufferR)
>>>>>> wd1 <- dnearneigh(pts, bds[1], bds[2])
>>>>>> wd2 <- dnearneigh(pts, bds[2], bds[3])
>>>>>> wd3 <- dnearneigh(pts, bds[3], bds[4])
>>>>>> wd4 <- dnearneigh(pts, bds[4], bds[5])
>>>>>> sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
>>>>>> sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
>>>>>> sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
>>>>>> sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
>>>>>> sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
>>>>>> sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
>>>>>> sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
>>>>>> sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
>>>>>> data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3,
>>>> sn_band4))
>>>>>> class(data_out) <- "data.frame"
>>>>>> table(data_out$band)
>>>>>> data_out$ID_from <- projdata$t50_fid[data_out$from]
>>>>>> data_out$ID_to <- projdata$t50_fid[data_out$to]
>>>>>> data_out$elev_from <- projdata$elevation[data_out$from]
>>>>>> data_out$elev_to <- projdata$elevation[data_out$to]
>>>>>> str(data_out)
>>>>>>
>>>>>> The "spatial.neighbour" representation was that used in the S-Plus
>>>>>> SpatialStats module, with "from" and "to" columns, and here drops
>>>>>> no-neighbour cases gracefully. So listw2sn() comes in useful
>>>>>> for creating the output, and from there, just look-up in the
>>>>>> input data.frame. Observations here cannot be their own neighbours.
>>>>>>
>>>>>> It would be relevant to know why you need these, are you looking at
>>>>>> variogram clouds?
>>>>>>
>>>>>> Hope this clarifies,
>>>>>>
>>>>>> Roger
>>>>>>
>>>>>>>
>>>>>>> ---------
>>>>>>> Lom
>>>>>>>
>>>>>>> On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]>
>>>> wrote:
>>>>>>>
>>>>>>>> Roger's example works for me and gives a list of length 101. I did
>>>> have
>>>>>>>> some issues that were resolved by updating packages. I'm using R
>> 3.6.3
>>>>>> on
>>>>>>>> macOS 10.15.4. I also use rtree successfully on Windows 10 with R
>>>> 3.6.3.
>>>>>>>>
>>>>>>>> Kent
>>>>>>>>
>>>>>>>> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>>>>>>>
>>>>>>>>>> rtree uses Euclidean distance so the points should be in a
>>>> coordinate
>>>>>>>>>> system where this makes sense at least as a reasonable
>>>> approximation.
>>>>>>>>>
>>>>>>>>> I tried the original example:
>>>>>>>>>
>>>>>>>>> remotes::install_github("hunzikp/rtree")
>>>>>>>>> library(spData)
>>>>>>>>> library(sf)
>>>>>>>>> projdata<-st_transform(nz_height, 32759)
>>>>>>>>> library(rtree)
>>>>>>>>> pts <- st_coordinates(projdata)
>>>>>>>>> rt <- RTree(st_coordinates(projdata))
>>>>>>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>>>>>>>>> wd1 <- withinDistance(rt, pts, bufferR[1])
>>>>>>>>>
>>>>>>>>> but unfortunately failed (maybe newer Boost headers than yours?):
>>>>>>>>>
>>>>>>>>> Error in UseMethod("withinDistance", rTree) :
>>>>>>>>>    no applicable method for 'withinDistance' applied to an object
>> of
>>>>>>>>> class
>>>>>>>>> "c('list', 'RTree')"
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kent
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>>>>>>>>>
>>>>>>>>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
>>>>>>>>>>>>> From: Lom Navanyo <[hidden email]>
>>>>>>>>>>>>> To: [hidden email]
>>>>>>>>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of
>>>>>> neighboring
>>>>>>>>>>>>>         points within specified radii (distances) for each
>> point
>>>>>> in a
>>>>>>>>>>> given
>>>>>>>>>>>>>         points data set.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>> I have data set of about 3400 location points with which I am
>>>>>> trying
>>>>>>>>> to
>>>>>>>>>>>>> generate data of each point and their neighbors within defined
>>>>>> radii
>>>>>>>>>>> (eg,
>>>>>>>>>>>>> 0.25, 1, and 3 miles).
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The rtree package is very fast and memory-efficient for
>>>>>>>>> within-distance
>>>>>>>>>>>> calculations.
>>>>>>>>>>>> https://github.com/hunzikp/rtree
>>>>>>>>>>>
>>>>>>>>>>> Thanks! Does this also apply when the input points are in
>>>>>> geographical
>>>>>>>>>>> coordinates?
>>>>>>>>>>>
>>>>>>>>>>> Roger
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Kent Johnson
>>>>>>>>>>>> Cambridge, MA
>>>>>>>>>>>>
>>>>>>>>>>>>       [[alternative HTML version deleted]]
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> R-sig-Geo mailing list
>>>>>>>>>>>> [hidden email]
>>>>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Roger Bivand
>>>>>>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>>>>>>>> https://orcid.org/0000-0003-2392-6140
>>>>>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Roger Bivand
>>>>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>>>>>> https://orcid.org/0000-0003-2392-6140
>>>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Roger Bivand
>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>>> https://orcid.org/0000-0003-2392-6140
>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Roger Bivand
>>>> Department of Economics, Norwegian School of Economics,
>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>> https://orcid.org/0000-0003-2392-6140
>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>
>>>
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: [hidden email]
>> https://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway

Re: How to efficiently generate data of neighboring points

15 hours 18 min ago
I fully agree with you and appreciate the listed benefits of not taking
things private. I was just trying to be sure the forum here is appropriate
and receptive of a beginner like me.

To be more explicit with regards to my observations, y is amount of water
withdrawal from wells and an important variable in x is (height of) water
level in the wells. These are end of year figures. I am using the
aggregations (sum for y and mean for water level) by band as spatial
neighborhood variables. There will be one or two indicator variables also
in x. I hope these do not
present additional hurdles.

 I am thinking Proximity is relevant in testing spatial
dependency/externality.

I will consider splm package  and the SLX model.

Thank you.
---------------
Lom

On Thu, Jun 4, 2020 at 2:52 PM Roger Bivand <[hidden email]> wrote:

> On Thu, 4 Jun 2020, Lom Navanyo wrote:
>
> > Thank you. Yes, the OLS is biased and my plan is to use a 2SLS approach.
> I
> > have a variable I intend to use as an IV for y.
> > I have seen a few papers use this approach. Will this approach not
> correct
> > for the endogeneity?
> >
> > Actually, I am not sure if this is a right forum or perhaps if it's
> > appropriate or acceptable to you to take this one-on-one with you for
> help:
>
> I do not offer private help. That would presuppose that one person has the
> answer. It would also presuppose that all exchanges are only read by the
> original poster and direct participants, while in fact others may join in,
> or follow a thread, or find the thread by searching: google supports the
> list:r-sig-geo search tag. If the thread goes private, that search is
> fruitless.
>
> > My model actually looks like this: y= f(y, x)  + e.
> > Aside the endogeneity of y (which I intend to instrument by another
> > variable z), there is simultaneity between y and x.
> > I intend to use the lag of x as instrument for x.  Given that I am
> seeking
> > to test spatial dependency, do you see some fatal flaws with my approach?
> >
>
> What is the support of your observations, point, or are they aggregations?
> Why may proximity make a difference - often, apparent spatial
> autocorrelation is caused by observing inappropriate entities, or by
> omitting covariates, or by using the wrong functional form.
>
>
> > I have also seen other empirical approaches like static and dynamic
> spatial
> > panel data modelling. I will be reviewing them also to see suitability
> for
> > my objective.
> > But, any further directions or suggestions are highly appreciated.
>
> If the data are spatial panel, you can look at the splm package.
> Personally, I have never found instruments any use at all, because the
> instruments are typically at best weak because of shared spatial processes
> with the response, unless the model is really well specified from known
> theory. In space, almost everything is close to endogeneous unless the
> opposite is demonstrated. So causal relationships are less worthwhile,
> because they are at best conditional on omitted variables and
> autocorrelation engendered by the choice of observational entities.
>
> Further, because spatial processes are driven by the inverse matrix of the
> input graph of proximate neighbours (the covariance matrix of the spatial
> process), you don't need to start from more than the first order
> neighbours. Maybe your x has the same spatial pattern as y, so that the
> residuals are white noise with no spatial structure.
>
> Recently, analysts prefer to start with the SLX model (Halleck Vega &
> Elhorst 2015 and others), so that might be worth exploring. If only the
> direct impacts seem important, OLS may be enough.
>
> Hope this helps,
>
> Roger
>
> >
> > Thanks,
> > -------------------
> > Lom
> >
> >
> >
> > On Thu, Jun 4, 2020 at 3:48 AM Roger Bivand <[hidden email]> wrote:
> >
> >> On Thu, 4 Jun 2020, Lom Navanyo wrote:
> >>
> >>> Thank you very much for your support. This gives me what I need and I
> >> must
> >>> say listw2sn() is really great.
> >>>
> >>> Why do I need the data in the format as in dataout? I am trying to test
> >>> spatial dependence (or neighborhood effect) by running a regression
> >>> model that entails pop_size_it = beta_1*sum of pop_size of point i's
> >>> neighbors within a specified radius. So my plan is to get the neighbors
> >>> for each focal point as per the specified bands and their attributes
> (eg
> >>> pop_size) so I can can add them (attribute) by the bands.
> >>
> >> Thanks, clarifies a good deal. Maybe look at the original localG
> articles
> >> for exploring distance relationships (Getis and Ord looked at HIV/AIDS);
> >> ?spdep::localG or
> https://r-spatial.github.io/spdep/reference/localG.html.
> >>
> >> Further note at OLS is biased as you have y = f(y) + e, so y on both
> >> sides. The nearest equivalent for a single band is
> spatialreg::lagsarlm()
> >> with listw=nb2listw(wd1, style="B") to get the neighbour sums through
> the
> >> weights matrix. So both your betas and their standard errors are
> unusable,
> >> I'm afraid. You are actually very much closer to ordinary kriging,
> looking
> >> at the way in which distance attenuates the correlation in value of
> >> proximate observations.
> >>
> >> Hope this clarifies,
> >>
> >> Roger
> >>
> >>>
> >>> I am totally new to the area of spatial econometrics, so I am taking
> >> things
> >>> one step at a time. Some readings suggest I may need distance matrix or
> >>> weight matrix but for now I think I should try the current approach.
> >>>
> >>> Thank you.
> >>>
> >>> -------------
> >>> Lom
> >>>
> >>> On Wed, Jun 3, 2020 at 8:18 AM Roger Bivand <[hidden email]>
> wrote:
> >>>
> >>>> On Wed, 3 Jun 2020, Lom Navanyo wrote:
> >>>>
> >>>>> I had the errors with rtree using R 3.6.3. I have since changed to R
> >>>> 4.0.0
> >>>>> but I got the same error.
> >>>>>
> >>>>> And  yes, for Roger's example, I have the objects wd1, ... wd4, all
> >> with
> >>>>> length 101. I think my difficulty is my inability to output the list
> >>>>> detailing the point IDs t50_fid.
> >>>>
> >>>> library(spData)
> >>>> library(sf)
> >>>> projdata<-st_transform(nz_height, 32759)
> >>>> pts <- st_coordinates(projdata)
> >>>> library(spdep)
> >>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> >>>> bds <- c(0, bufferR)
> >>>> wd1 <- dnearneigh(pts, bds[1], bds[2])
> >>>> wd2 <- dnearneigh(pts, bds[2], bds[3])
> >>>> wd3 <- dnearneigh(pts, bds[3], bds[4])
> >>>> wd4 <- dnearneigh(pts, bds[4], bds[5])
> >>>> sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
> >>>> sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
> >>>> sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
> >>>> sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
> >>>> sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
> >>>> sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
> >>>> sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
> >>>> sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
> >>>> data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3,
> >> sn_band4))
> >>>> class(data_out) <- "data.frame"
> >>>> table(data_out$band)
> >>>> data_out$ID_from <- projdata$t50_fid[data_out$from]
> >>>> data_out$ID_to <- projdata$t50_fid[data_out$to]
> >>>> data_out$elev_from <- projdata$elevation[data_out$from]
> >>>> data_out$elev_to <- projdata$elevation[data_out$to]
> >>>> str(data_out)
> >>>>
> >>>> The "spatial.neighbour" representation was that used in the S-Plus
> >>>> SpatialStats module, with "from" and "to" columns, and here drops
> >>>> no-neighbour cases gracefully. So listw2sn() comes in useful
> >>>> for creating the output, and from there, just look-up in the
> >>>> input data.frame. Observations here cannot be their own neighbours.
> >>>>
> >>>> It would be relevant to know why you need these, are you looking at
> >>>> variogram clouds?
> >>>>
> >>>> Hope this clarifies,
> >>>>
> >>>> Roger
> >>>>
> >>>>>
> >>>>> ---------
> >>>>> Lom
> >>>>>
> >>>>> On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]>
> >> wrote:
> >>>>>
> >>>>>> Roger's example works for me and gives a list of length 101. I did
> >> have
> >>>>>> some issues that were resolved by updating packages. I'm using R
> 3.6.3
> >>>> on
> >>>>>> macOS 10.15.4. I also use rtree successfully on Windows 10 with R
> >> 3.6.3.
> >>>>>>
> >>>>>> Kent
> >>>>>>
> >>>>>> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]>
> >>>> wrote:
> >>>>>>
> >>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>>>>>
> >>>>>>>> rtree uses Euclidean distance so the points should be in a
> >> coordinate
> >>>>>>>> system where this makes sense at least as a reasonable
> >> approximation.
> >>>>>>>
> >>>>>>> I tried the original example:
> >>>>>>>
> >>>>>>> remotes::install_github("hunzikp/rtree")
> >>>>>>> library(spData)
> >>>>>>> library(sf)
> >>>>>>> projdata<-st_transform(nz_height, 32759)
> >>>>>>> library(rtree)
> >>>>>>> pts <- st_coordinates(projdata)
> >>>>>>> rt <- RTree(st_coordinates(projdata))
> >>>>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> >>>>>>> wd1 <- withinDistance(rt, pts, bufferR[1])
> >>>>>>>
> >>>>>>> but unfortunately failed (maybe newer Boost headers than yours?):
> >>>>>>>
> >>>>>>> Error in UseMethod("withinDistance", rTree) :
> >>>>>>>    no applicable method for 'withinDistance' applied to an object
> of
> >>>>>>> class
> >>>>>>> "c('list', 'RTree')"
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Kent
> >>>>>>>>
> >>>>>>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>>>>>>>
> >>>>>>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
> >>>>>>>>>>> From: Lom Navanyo <[hidden email]>
> >>>>>>>>>>> To: [hidden email]
> >>>>>>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of
> >>>> neighboring
> >>>>>>>>>>>         points within specified radii (distances) for each
> point
> >>>> in a
> >>>>>>>>> given
> >>>>>>>>>>>         points data set.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Hello,
> >>>>>>>>>>> I have data set of about 3400 location points with which I am
> >>>> trying
> >>>>>>> to
> >>>>>>>>>>> generate data of each point and their neighbors within defined
> >>>> radii
> >>>>>>>>> (eg,
> >>>>>>>>>>> 0.25, 1, and 3 miles).
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> The rtree package is very fast and memory-efficient for
> >>>>>>> within-distance
> >>>>>>>>>> calculations.
> >>>>>>>>>> https://github.com/hunzikp/rtree
> >>>>>>>>>
> >>>>>>>>> Thanks! Does this also apply when the input points are in
> >>>> geographical
> >>>>>>>>> coordinates?
> >>>>>>>>>
> >>>>>>>>> Roger
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Kent Johnson
> >>>>>>>>>> Cambridge, MA
> >>>>>>>>>>
> >>>>>>>>>>       [[alternative HTML version deleted]]
> >>>>>>>>>>
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> R-sig-Geo mailing list
> >>>>>>>>>> [hidden email]
> >>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Roger Bivand
> >>>>>>>>> Department of Economics, Norwegian School of Economics,
> >>>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>>>>>> https://orcid.org/0000-0003-2392-6140
> >>>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Roger Bivand
> >>>>>>> Department of Economics, Norwegian School of Economics,
> >>>>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>>>> https://orcid.org/0000-0003-2392-6140
> >>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> Roger Bivand
> >>>> Department of Economics, Norwegian School of Economics,
> >>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>> https://orcid.org/0000-0003-2392-6140
> >>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>
> >>>
> >>
> >> --
> >> Roger Bivand
> >> Department of Economics, Norwegian School of Economics,
> >> Helleveien 30, N-5045 Bergen, Norway.
> >> voice: +47 55 95 93 55; e-mail: [hidden email]
> >> https://orcid.org/0000-0003-2392-6140
> >> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

GLM model with spatialspillover on categorical variables

18 hours 16 min ago
I did a regression analysis with categorical data with a glm model
approach, which worked fine. I have longitude and latitude coordinates for
each observation and I want to add their geographic spillover effect to the
model.

My sample data is structured:

Index DV IVI IVII IVIII IVIV Long Lat
 1  0  2  1  3  -12  -17.8  12
 2  0  1  1  6  112  11  -122
 3  1  3  6  1  91  57  53

with regression eq. DV ~ IVI + IVII + IVIII + IVIV

That mentioned, I assume that the nearer regions are, the more it may
influence my dependant variable. I found several approaches for spatial
regression models, but not for categorical data. I tried to use existing
libraries and functions, such as spdep's lagsarlm, glmmfields, spatialreg,
gstat, geoRglm and many more (I used this list as a reference:
https://cran.r-project.org/web/views/Spatial.html ). For numeric values, I
am able to do spatial regression, but for categorical values, I struggle.
The data structure is the following:

library(dplyr)
data <- data %>%
  mutate(
    DV = as.factor(DV),
    IVI = as.factor(IVI),
    IVII = as.factor(IVII),
    IVIII = as.factor(IVIII),
    IVIV = as.numeric(IVIV),
    longitude = as.numeric(longitude),
    latitude = as.numeric(latitude)
  )

My dependant variable (0|1) as well as my independant variables are
categorical and it would be no use to transform them, of course. I want to
have an other glm model in the end, but with spatial spillover effects
included. The libraries I tested so far can't handle categorical data. Any
leads/ideas would be greatly appreciated.

Thanks a lot.

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: How to efficiently generate data of neighboring points

Thu, 06/04/2020 - 14:52
On Thu, 4 Jun 2020, Lom Navanyo wrote:

> Thank you. Yes, the OLS is biased and my plan is to use a 2SLS approach. I
> have a variable I intend to use as an IV for y.
> I have seen a few papers use this approach. Will this approach not correct
> for the endogeneity?
>
> Actually, I am not sure if this is a right forum or perhaps if it's
> appropriate or acceptable to you to take this one-on-one with you for help:

I do not offer private help. That would presuppose that one person has the
answer. It would also presuppose that all exchanges are only read by the
original poster and direct participants, while in fact others may join in,
or follow a thread, or find the thread by searching: google supports the
list:r-sig-geo search tag. If the thread goes private, that search is
fruitless.

> My model actually looks like this: y= f(y, x)  + e.
> Aside the endogeneity of y (which I intend to instrument by another
> variable z), there is simultaneity between y and x.
> I intend to use the lag of x as instrument for x.  Given that I am seeking
> to test spatial dependency, do you see some fatal flaws with my approach?
>

What is the support of your observations, point, or are they aggregations?
Why may proximity make a difference - often, apparent spatial
autocorrelation is caused by observing inappropriate entities, or by
omitting covariates, or by using the wrong functional form.


> I have also seen other empirical approaches like static and dynamic spatial
> panel data modelling. I will be reviewing them also to see suitability for
> my objective.
> But, any further directions or suggestions are highly appreciated.

If the data are spatial panel, you can look at the splm package.
Personally, I have never found instruments any use at all, because the
instruments are typically at best weak because of shared spatial processes
with the response, unless the model is really well specified from known
theory. In space, almost everything is close to endogeneous unless the
opposite is demonstrated. So causal relationships are less worthwhile,
because they are at best conditional on omitted variables and
autocorrelation engendered by the choice of observational entities.

Further, because spatial processes are driven by the inverse matrix of the
input graph of proximate neighbours (the covariance matrix of the spatial
process), you don't need to start from more than the first order
neighbours. Maybe your x has the same spatial pattern as y, so that the
residuals are white noise with no spatial structure.

Recently, analysts prefer to start with the SLX model (Halleck Vega &
Elhorst 2015 and others), so that might be worth exploring. If only the
direct impacts seem important, OLS may be enough.

Hope this helps,

Roger

>
> Thanks,
> -------------------
> Lom
>
>
>
> On Thu, Jun 4, 2020 at 3:48 AM Roger Bivand <[hidden email]> wrote:
>
>> On Thu, 4 Jun 2020, Lom Navanyo wrote:
>>
>>> Thank you very much for your support. This gives me what I need and I
>> must
>>> say listw2sn() is really great.
>>>
>>> Why do I need the data in the format as in dataout? I am trying to test
>>> spatial dependence (or neighborhood effect) by running a regression
>>> model that entails pop_size_it = beta_1*sum of pop_size of point i's
>>> neighbors within a specified radius. So my plan is to get the neighbors
>>> for each focal point as per the specified bands and their attributes (eg
>>> pop_size) so I can can add them (attribute) by the bands.
>>
>> Thanks, clarifies a good deal. Maybe look at the original localG articles
>> for exploring distance relationships (Getis and Ord looked at HIV/AIDS);
>> ?spdep::localG or https://r-spatial.github.io/spdep/reference/localG.html.
>>
>> Further note at OLS is biased as you have y = f(y) + e, so y on both
>> sides. The nearest equivalent for a single band is spatialreg::lagsarlm()
>> with listw=nb2listw(wd1, style="B") to get the neighbour sums through the
>> weights matrix. So both your betas and their standard errors are unusable,
>> I'm afraid. You are actually very much closer to ordinary kriging, looking
>> at the way in which distance attenuates the correlation in value of
>> proximate observations.
>>
>> Hope this clarifies,
>>
>> Roger
>>
>>>
>>> I am totally new to the area of spatial econometrics, so I am taking
>> things
>>> one step at a time. Some readings suggest I may need distance matrix or
>>> weight matrix but for now I think I should try the current approach.
>>>
>>> Thank you.
>>>
>>> -------------
>>> Lom
>>>
>>> On Wed, Jun 3, 2020 at 8:18 AM Roger Bivand <[hidden email]> wrote:
>>>
>>>> On Wed, 3 Jun 2020, Lom Navanyo wrote:
>>>>
>>>>> I had the errors with rtree using R 3.6.3. I have since changed to R
>>>> 4.0.0
>>>>> but I got the same error.
>>>>>
>>>>> And  yes, for Roger's example, I have the objects wd1, ... wd4, all
>> with
>>>>> length 101. I think my difficulty is my inability to output the list
>>>>> detailing the point IDs t50_fid.
>>>>
>>>> library(spData)
>>>> library(sf)
>>>> projdata<-st_transform(nz_height, 32759)
>>>> pts <- st_coordinates(projdata)
>>>> library(spdep)
>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>>>> bds <- c(0, bufferR)
>>>> wd1 <- dnearneigh(pts, bds[1], bds[2])
>>>> wd2 <- dnearneigh(pts, bds[2], bds[3])
>>>> wd3 <- dnearneigh(pts, bds[3], bds[4])
>>>> wd4 <- dnearneigh(pts, bds[4], bds[5])
>>>> sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
>>>> sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
>>>> sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
>>>> sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
>>>> sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
>>>> sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
>>>> sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
>>>> sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
>>>> data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3,
>> sn_band4))
>>>> class(data_out) <- "data.frame"
>>>> table(data_out$band)
>>>> data_out$ID_from <- projdata$t50_fid[data_out$from]
>>>> data_out$ID_to <- projdata$t50_fid[data_out$to]
>>>> data_out$elev_from <- projdata$elevation[data_out$from]
>>>> data_out$elev_to <- projdata$elevation[data_out$to]
>>>> str(data_out)
>>>>
>>>> The "spatial.neighbour" representation was that used in the S-Plus
>>>> SpatialStats module, with "from" and "to" columns, and here drops
>>>> no-neighbour cases gracefully. So listw2sn() comes in useful
>>>> for creating the output, and from there, just look-up in the
>>>> input data.frame. Observations here cannot be their own neighbours.
>>>>
>>>> It would be relevant to know why you need these, are you looking at
>>>> variogram clouds?
>>>>
>>>> Hope this clarifies,
>>>>
>>>> Roger
>>>>
>>>>>
>>>>> ---------
>>>>> Lom
>>>>>
>>>>> On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]>
>> wrote:
>>>>>
>>>>>> Roger's example works for me and gives a list of length 101. I did
>> have
>>>>>> some issues that were resolved by updating packages. I'm using R 3.6.3
>>>> on
>>>>>> macOS 10.15.4. I also use rtree successfully on Windows 10 with R
>> 3.6.3.
>>>>>>
>>>>>> Kent
>>>>>>
>>>>>> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]>
>>>> wrote:
>>>>>>
>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>>>>>
>>>>>>>> rtree uses Euclidean distance so the points should be in a
>> coordinate
>>>>>>>> system where this makes sense at least as a reasonable
>> approximation.
>>>>>>>
>>>>>>> I tried the original example:
>>>>>>>
>>>>>>> remotes::install_github("hunzikp/rtree")
>>>>>>> library(spData)
>>>>>>> library(sf)
>>>>>>> projdata<-st_transform(nz_height, 32759)
>>>>>>> library(rtree)
>>>>>>> pts <- st_coordinates(projdata)
>>>>>>> rt <- RTree(st_coordinates(projdata))
>>>>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>>>>>>> wd1 <- withinDistance(rt, pts, bufferR[1])
>>>>>>>
>>>>>>> but unfortunately failed (maybe newer Boost headers than yours?):
>>>>>>>
>>>>>>> Error in UseMethod("withinDistance", rTree) :
>>>>>>>    no applicable method for 'withinDistance' applied to an object of
>>>>>>> class
>>>>>>> "c('list', 'RTree')"
>>>>>>>
>>>>>>>>
>>>>>>>> Kent
>>>>>>>>
>>>>>>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>>>>>>>
>>>>>>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
>>>>>>>>>>> From: Lom Navanyo <[hidden email]>
>>>>>>>>>>> To: [hidden email]
>>>>>>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of
>>>> neighboring
>>>>>>>>>>>         points within specified radii (distances) for each point
>>>> in a
>>>>>>>>> given
>>>>>>>>>>>         points data set.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>> I have data set of about 3400 location points with which I am
>>>> trying
>>>>>>> to
>>>>>>>>>>> generate data of each point and their neighbors within defined
>>>> radii
>>>>>>>>> (eg,
>>>>>>>>>>> 0.25, 1, and 3 miles).
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The rtree package is very fast and memory-efficient for
>>>>>>> within-distance
>>>>>>>>>> calculations.
>>>>>>>>>> https://github.com/hunzikp/rtree
>>>>>>>>>
>>>>>>>>> Thanks! Does this also apply when the input points are in
>>>> geographical
>>>>>>>>> coordinates?
>>>>>>>>>
>>>>>>>>> Roger
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kent Johnson
>>>>>>>>>> Cambridge, MA
>>>>>>>>>>
>>>>>>>>>>       [[alternative HTML version deleted]]
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> R-sig-Geo mailing list
>>>>>>>>>> [hidden email]
>>>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Roger Bivand
>>>>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>>>>>> https://orcid.org/0000-0003-2392-6140
>>>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Roger Bivand
>>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>>>> https://orcid.org/0000-0003-2392-6140
>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> --
>>>> Roger Bivand
>>>> Department of Economics, Norwegian School of Economics,
>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>> https://orcid.org/0000-0003-2392-6140
>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>
>>>
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: [hidden email]
>> https://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway

Re: How to efficiently generate data of neighboring points

Thu, 06/04/2020 - 13:50
Thank you. Yes, the OLS is biased and my plan is to use a 2SLS approach. I
have a variable I intend to use as an IV for y.
I have seen a few papers use this approach. Will this approach not correct
for the endogeneity?

Actually, I am not sure if this is a right forum or perhaps if it's
appropriate or acceptable to you to take this one-on-one with you for help:
My model actually looks like this: y= f(y, x)  + e.
Aside the endogeneity of y (which I intend to instrument by another
variable z), there is simultaneity between y and x.
I intend to use the lag of x as instrument for x.  Given that I am seeking
to test spatial dependency, do you see some fatal flaws with my approach?

I have also seen other empirical approaches like static and dynamic spatial
panel data modelling. I will be reviewing them also to see suitability for
my objective.
But, any further directions or suggestions are highly appreciated.

Thanks,
-------------------
Lom



On Thu, Jun 4, 2020 at 3:48 AM Roger Bivand <[hidden email]> wrote:

> On Thu, 4 Jun 2020, Lom Navanyo wrote:
>
> > Thank you very much for your support. This gives me what I need and I
> must
> > say listw2sn() is really great.
> >
> > Why do I need the data in the format as in dataout? I am trying to test
> > spatial dependence (or neighborhood effect) by running a regression
> > model that entails pop_size_it = beta_1*sum of pop_size of point i's
> > neighbors within a specified radius. So my plan is to get the neighbors
> > for each focal point as per the specified bands and their attributes (eg
> > pop_size) so I can can add them (attribute) by the bands.
>
> Thanks, clarifies a good deal. Maybe look at the original localG articles
> for exploring distance relationships (Getis and Ord looked at HIV/AIDS);
> ?spdep::localG or https://r-spatial.github.io/spdep/reference/localG.html.
>
> Further note at OLS is biased as you have y = f(y) + e, so y on both
> sides. The nearest equivalent for a single band is spatialreg::lagsarlm()
> with listw=nb2listw(wd1, style="B") to get the neighbour sums through the
> weights matrix. So both your betas and their standard errors are unusable,
> I'm afraid. You are actually very much closer to ordinary kriging, looking
> at the way in which distance attenuates the correlation in value of
> proximate observations.
>
> Hope this clarifies,
>
> Roger
>
> >
> > I am totally new to the area of spatial econometrics, so I am taking
> things
> > one step at a time. Some readings suggest I may need distance matrix or
> > weight matrix but for now I think I should try the current approach.
> >
> > Thank you.
> >
> > -------------
> > Lom
> >
> > On Wed, Jun 3, 2020 at 8:18 AM Roger Bivand <[hidden email]> wrote:
> >
> >> On Wed, 3 Jun 2020, Lom Navanyo wrote:
> >>
> >>> I had the errors with rtree using R 3.6.3. I have since changed to R
> >> 4.0.0
> >>> but I got the same error.
> >>>
> >>> And  yes, for Roger's example, I have the objects wd1, ... wd4, all
> with
> >>> length 101. I think my difficulty is my inability to output the list
> >>> detailing the point IDs t50_fid.
> >>
> >> library(spData)
> >> library(sf)
> >> projdata<-st_transform(nz_height, 32759)
> >> pts <- st_coordinates(projdata)
> >> library(spdep)
> >> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> >> bds <- c(0, bufferR)
> >> wd1 <- dnearneigh(pts, bds[1], bds[2])
> >> wd2 <- dnearneigh(pts, bds[2], bds[3])
> >> wd3 <- dnearneigh(pts, bds[3], bds[4])
> >> wd4 <- dnearneigh(pts, bds[4], bds[5])
> >> sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
> >> sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
> >> sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
> >> sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
> >> sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
> >> sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
> >> sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
> >> sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
> >> data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3,
> sn_band4))
> >> class(data_out) <- "data.frame"
> >> table(data_out$band)
> >> data_out$ID_from <- projdata$t50_fid[data_out$from]
> >> data_out$ID_to <- projdata$t50_fid[data_out$to]
> >> data_out$elev_from <- projdata$elevation[data_out$from]
> >> data_out$elev_to <- projdata$elevation[data_out$to]
> >> str(data_out)
> >>
> >> The "spatial.neighbour" representation was that used in the S-Plus
> >> SpatialStats module, with "from" and "to" columns, and here drops
> >> no-neighbour cases gracefully. So listw2sn() comes in useful
> >> for creating the output, and from there, just look-up in the
> >> input data.frame. Observations here cannot be their own neighbours.
> >>
> >> It would be relevant to know why you need these, are you looking at
> >> variogram clouds?
> >>
> >> Hope this clarifies,
> >>
> >> Roger
> >>
> >>>
> >>> ---------
> >>> Lom
> >>>
> >>> On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]>
> wrote:
> >>>
> >>>> Roger's example works for me and gives a list of length 101. I did
> have
> >>>> some issues that were resolved by updating packages. I'm using R 3.6.3
> >> on
> >>>> macOS 10.15.4. I also use rtree successfully on Windows 10 with R
> 3.6.3.
> >>>>
> >>>> Kent
> >>>>
> >>>> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]>
> >> wrote:
> >>>>
> >>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>>>
> >>>>>> rtree uses Euclidean distance so the points should be in a
> coordinate
> >>>>>> system where this makes sense at least as a reasonable
> approximation.
> >>>>>
> >>>>> I tried the original example:
> >>>>>
> >>>>> remotes::install_github("hunzikp/rtree")
> >>>>> library(spData)
> >>>>> library(sf)
> >>>>> projdata<-st_transform(nz_height, 32759)
> >>>>> library(rtree)
> >>>>> pts <- st_coordinates(projdata)
> >>>>> rt <- RTree(st_coordinates(projdata))
> >>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> >>>>> wd1 <- withinDistance(rt, pts, bufferR[1])
> >>>>>
> >>>>> but unfortunately failed (maybe newer Boost headers than yours?):
> >>>>>
> >>>>> Error in UseMethod("withinDistance", rTree) :
> >>>>>    no applicable method for 'withinDistance' applied to an object of
> >>>>> class
> >>>>> "c('list', 'RTree')"
> >>>>>
> >>>>>>
> >>>>>> Kent
> >>>>>>
> >>>>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
> >>>>> wrote:
> >>>>>>
> >>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>>>>>
> >>>>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
> >>>>>>>>> From: Lom Navanyo <[hidden email]>
> >>>>>>>>> To: [hidden email]
> >>>>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of
> >> neighboring
> >>>>>>>>>         points within specified radii (distances) for each point
> >> in a
> >>>>>>> given
> >>>>>>>>>         points data set.
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Hello,
> >>>>>>>>> I have data set of about 3400 location points with which I am
> >> trying
> >>>>> to
> >>>>>>>>> generate data of each point and their neighbors within defined
> >> radii
> >>>>>>> (eg,
> >>>>>>>>> 0.25, 1, and 3 miles).
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> The rtree package is very fast and memory-efficient for
> >>>>> within-distance
> >>>>>>>> calculations.
> >>>>>>>> https://github.com/hunzikp/rtree
> >>>>>>>
> >>>>>>> Thanks! Does this also apply when the input points are in
> >> geographical
> >>>>>>> coordinates?
> >>>>>>>
> >>>>>>> Roger
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Kent Johnson
> >>>>>>>> Cambridge, MA
> >>>>>>>>
> >>>>>>>>       [[alternative HTML version deleted]]
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> R-sig-Geo mailing list
> >>>>>>>> [hidden email]
> >>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Roger Bivand
> >>>>>>> Department of Economics, Norwegian School of Economics,
> >>>>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>>>> https://orcid.org/0000-0003-2392-6140
> >>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> Roger Bivand
> >>>>> Department of Economics, Norwegian School of Economics,
> >>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>> https://orcid.org/0000-0003-2392-6140
> >>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>
> >>>>
> >>>
> >>
> >> --
> >> Roger Bivand
> >> Department of Economics, Norwegian School of Economics,
> >> Helleveien 30, N-5045 Bergen, Norway.
> >> voice: +47 55 95 93 55; e-mail: [hidden email]
> >> https://orcid.org/0000-0003-2392-6140
> >> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: How to efficiently generate data of neighboring points

Thu, 06/04/2020 - 03:48
On Thu, 4 Jun 2020, Lom Navanyo wrote:

> Thank you very much for your support. This gives me what I need and I must
> say listw2sn() is really great.
>
> Why do I need the data in the format as in dataout? I am trying to test
> spatial dependence (or neighborhood effect) by running a regression
> model that entails pop_size_it = beta_1*sum of pop_size of point i's
> neighbors within a specified radius. So my plan is to get the neighbors
> for each focal point as per the specified bands and their attributes (eg
> pop_size) so I can can add them (attribute) by the bands.

Thanks, clarifies a good deal. Maybe look at the original localG articles
for exploring distance relationships (Getis and Ord looked at HIV/AIDS);
?spdep::localG or https://r-spatial.github.io/spdep/reference/localG.html.

Further note at OLS is biased as you have y = f(y) + e, so y on both
sides. The nearest equivalent for a single band is spatialreg::lagsarlm()
with listw=nb2listw(wd1, style="B") to get the neighbour sums through the
weights matrix. So both your betas and their standard errors are unusable,
I'm afraid. You are actually very much closer to ordinary kriging, looking
at the way in which distance attenuates the correlation in value of
proximate observations.

Hope this clarifies,

Roger

>
> I am totally new to the area of spatial econometrics, so I am taking things
> one step at a time. Some readings suggest I may need distance matrix or
> weight matrix but for now I think I should try the current approach.
>
> Thank you.
>
> -------------
> Lom
>
> On Wed, Jun 3, 2020 at 8:18 AM Roger Bivand <[hidden email]> wrote:
>
>> On Wed, 3 Jun 2020, Lom Navanyo wrote:
>>
>>> I had the errors with rtree using R 3.6.3. I have since changed to R
>> 4.0.0
>>> but I got the same error.
>>>
>>> And  yes, for Roger's example, I have the objects wd1, ... wd4, all with
>>> length 101. I think my difficulty is my inability to output the list
>>> detailing the point IDs t50_fid.
>>
>> library(spData)
>> library(sf)
>> projdata<-st_transform(nz_height, 32759)
>> pts <- st_coordinates(projdata)
>> library(spdep)
>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>> bds <- c(0, bufferR)
>> wd1 <- dnearneigh(pts, bds[1], bds[2])
>> wd2 <- dnearneigh(pts, bds[2], bds[3])
>> wd3 <- dnearneigh(pts, bds[3], bds[4])
>> wd4 <- dnearneigh(pts, bds[4], bds[5])
>> sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
>> sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
>> sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
>> sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
>> sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
>> sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
>> sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
>> sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
>> data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3, sn_band4))
>> class(data_out) <- "data.frame"
>> table(data_out$band)
>> data_out$ID_from <- projdata$t50_fid[data_out$from]
>> data_out$ID_to <- projdata$t50_fid[data_out$to]
>> data_out$elev_from <- projdata$elevation[data_out$from]
>> data_out$elev_to <- projdata$elevation[data_out$to]
>> str(data_out)
>>
>> The "spatial.neighbour" representation was that used in the S-Plus
>> SpatialStats module, with "from" and "to" columns, and here drops
>> no-neighbour cases gracefully. So listw2sn() comes in useful
>> for creating the output, and from there, just look-up in the
>> input data.frame. Observations here cannot be their own neighbours.
>>
>> It would be relevant to know why you need these, are you looking at
>> variogram clouds?
>>
>> Hope this clarifies,
>>
>> Roger
>>
>>>
>>> ---------
>>> Lom
>>>
>>> On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]> wrote:
>>>
>>>> Roger's example works for me and gives a list of length 101. I did have
>>>> some issues that were resolved by updating packages. I'm using R 3.6.3
>> on
>>>> macOS 10.15.4. I also use rtree successfully on Windows 10 with R 3.6.3.
>>>>
>>>> Kent
>>>>
>>>> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]>
>> wrote:
>>>>
>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>>>
>>>>>> rtree uses Euclidean distance so the points should be in a coordinate
>>>>>> system where this makes sense at least as a reasonable approximation.
>>>>>
>>>>> I tried the original example:
>>>>>
>>>>> remotes::install_github("hunzikp/rtree")
>>>>> library(spData)
>>>>> library(sf)
>>>>> projdata<-st_transform(nz_height, 32759)
>>>>> library(rtree)
>>>>> pts <- st_coordinates(projdata)
>>>>> rt <- RTree(st_coordinates(projdata))
>>>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>>>>> wd1 <- withinDistance(rt, pts, bufferR[1])
>>>>>
>>>>> but unfortunately failed (maybe newer Boost headers than yours?):
>>>>>
>>>>> Error in UseMethod("withinDistance", rTree) :
>>>>>    no applicable method for 'withinDistance' applied to an object of
>>>>> class
>>>>> "c('list', 'RTree')"
>>>>>
>>>>>>
>>>>>> Kent
>>>>>>
>>>>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>>>>>
>>>>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
>>>>>>>>> From: Lom Navanyo <[hidden email]>
>>>>>>>>> To: [hidden email]
>>>>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of
>> neighboring
>>>>>>>>>         points within specified radii (distances) for each point
>> in a
>>>>>>> given
>>>>>>>>>         points data set.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>> I have data set of about 3400 location points with which I am
>> trying
>>>>> to
>>>>>>>>> generate data of each point and their neighbors within defined
>> radii
>>>>>>> (eg,
>>>>>>>>> 0.25, 1, and 3 miles).
>>>>>>>>>
>>>>>>>>
>>>>>>>> The rtree package is very fast and memory-efficient for
>>>>> within-distance
>>>>>>>> calculations.
>>>>>>>> https://github.com/hunzikp/rtree
>>>>>>>
>>>>>>> Thanks! Does this also apply when the input points are in
>> geographical
>>>>>>> coordinates?
>>>>>>>
>>>>>>> Roger
>>>>>>>
>>>>>>>>
>>>>>>>> Kent Johnson
>>>>>>>> Cambridge, MA
>>>>>>>>
>>>>>>>>       [[alternative HTML version deleted]]
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> R-sig-Geo mailing list
>>>>>>>> [hidden email]
>>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Roger Bivand
>>>>>>> Department of Economics, Norwegian School of Economics,
>>>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>>>> https://orcid.org/0000-0003-2392-6140
>>>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Roger Bivand
>>>>> Department of Economics, Norwegian School of Economics,
>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>> https://orcid.org/0000-0003-2392-6140
>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>
>>>>
>>>
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: [hidden email]
>> https://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway

Re: How to efficiently generate data of neighboring points

Wed, 06/03/2020 - 22:44
Thank you very much for your support. This gives me what I need and I must
say listw2sn() is really great.

Why do I need the data in the format as in dataout? I am trying to test
spatial dependence (or neighborhood effect) by running a regression model
that entails pop_size_it = beta_1*sum of pop_size of point i's neighbors
within a specified radius. So my plan is to get the neighbors for each
focal point
as per the specified bands and their attributes (eg pop_size) so I can can
add them (attribute) by the bands.

I am totally new to the area of spatial econometrics, so I am taking things
one step at a time. Some readings suggest I may need distance matrix or
weight matrix but for now I think I should try the current approach.

Thank you.

-------------
Lom

On Wed, Jun 3, 2020 at 8:18 AM Roger Bivand <[hidden email]> wrote:

> On Wed, 3 Jun 2020, Lom Navanyo wrote:
>
> > I had the errors with rtree using R 3.6.3. I have since changed to R
> 4.0.0
> > but I got the same error.
> >
> > And  yes, for Roger's example, I have the objects wd1, ... wd4, all with
> > length 101. I think my difficulty is my inability to output the list
> > detailing the point IDs t50_fid.
>
> library(spData)
> library(sf)
> projdata<-st_transform(nz_height, 32759)
> pts <- st_coordinates(projdata)
> library(spdep)
> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> bds <- c(0, bufferR)
> wd1 <- dnearneigh(pts, bds[1], bds[2])
> wd2 <- dnearneigh(pts, bds[2], bds[3])
> wd3 <- dnearneigh(pts, bds[3], bds[4])
> wd4 <- dnearneigh(pts, bds[4], bds[5])
> sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
> sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
> sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
> sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
> sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
> sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
> sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
> sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
> data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3, sn_band4))
> class(data_out) <- "data.frame"
> table(data_out$band)
> data_out$ID_from <- projdata$t50_fid[data_out$from]
> data_out$ID_to <- projdata$t50_fid[data_out$to]
> data_out$elev_from <- projdata$elevation[data_out$from]
> data_out$elev_to <- projdata$elevation[data_out$to]
> str(data_out)
>
> The "spatial.neighbour" representation was that used in the S-Plus
> SpatialStats module, with "from" and "to" columns, and here drops
> no-neighbour cases gracefully. So listw2sn() comes in useful
> for creating the output, and from there, just look-up in the
> input data.frame. Observations here cannot be their own neighbours.
>
> It would be relevant to know why you need these, are you looking at
> variogram clouds?
>
> Hope this clarifies,
>
> Roger
>
> >
> > ---------
> > Lom
> >
> > On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]> wrote:
> >
> >> Roger's example works for me and gives a list of length 101. I did have
> >> some issues that were resolved by updating packages. I'm using R 3.6.3
> on
> >> macOS 10.15.4. I also use rtree successfully on Windows 10 with R 3.6.3.
> >>
> >> Kent
> >>
> >> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]>
> wrote:
> >>
> >>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>
> >>>> rtree uses Euclidean distance so the points should be in a coordinate
> >>>> system where this makes sense at least as a reasonable approximation.
> >>>
> >>> I tried the original example:
> >>>
> >>> remotes::install_github("hunzikp/rtree")
> >>> library(spData)
> >>> library(sf)
> >>> projdata<-st_transform(nz_height, 32759)
> >>> library(rtree)
> >>> pts <- st_coordinates(projdata)
> >>> rt <- RTree(st_coordinates(projdata))
> >>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
> >>> wd1 <- withinDistance(rt, pts, bufferR[1])
> >>>
> >>> but unfortunately failed (maybe newer Boost headers than yours?):
> >>>
> >>> Error in UseMethod("withinDistance", rTree) :
> >>>    no applicable method for 'withinDistance' applied to an object of
> >>> class
> >>> "c('list', 'RTree')"
> >>>
> >>>>
> >>>> Kent
> >>>>
> >>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
> >>> wrote:
> >>>>
> >>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
> >>>>>
> >>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
> >>>>>>> From: Lom Navanyo <[hidden email]>
> >>>>>>> To: [hidden email]
> >>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of
> neighboring
> >>>>>>>         points within specified radii (distances) for each point
> in a
> >>>>> given
> >>>>>>>         points data set.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> Hello,
> >>>>>>> I have data set of about 3400 location points with which I am
> trying
> >>> to
> >>>>>>> generate data of each point and their neighbors within defined
> radii
> >>>>> (eg,
> >>>>>>> 0.25, 1, and 3 miles).
> >>>>>>>
> >>>>>>
> >>>>>> The rtree package is very fast and memory-efficient for
> >>> within-distance
> >>>>>> calculations.
> >>>>>> https://github.com/hunzikp/rtree
> >>>>>
> >>>>> Thanks! Does this also apply when the input points are in
> geographical
> >>>>> coordinates?
> >>>>>
> >>>>> Roger
> >>>>>
> >>>>>>
> >>>>>> Kent Johnson
> >>>>>> Cambridge, MA
> >>>>>>
> >>>>>>       [[alternative HTML version deleted]]
> >>>>>>
> >>>>>> _______________________________________________
> >>>>>> R-sig-Geo mailing list
> >>>>>> [hidden email]
> >>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> Roger Bivand
> >>>>> Department of Economics, Norwegian School of Economics,
> >>>>> Helleveien 30, N-5045 Bergen, Norway.
> >>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>>>> https://orcid.org/0000-0003-2392-6140
> >>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>>>
> >>>>
> >>>
> >>> --
> >>> Roger Bivand
> >>> Department of Economics, Norwegian School of Economics,
> >>> Helleveien 30, N-5045 Bergen, Norway.
> >>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>> https://orcid.org/0000-0003-2392-6140
> >>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>>
> >>
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: rgdal: About the new gdal3 and proj >=6 condition

Wed, 06/03/2020 - 12:42
I was trying from source, but now installed the binary.

Thank you

On Wed, Jun 3, 2020 at 3:30 PM Roger Bivand <[hidden email]> wrote:

> On Wed, 3 Jun 2020, Abdoulaye Sarr wrote:
>
> > Have a problem with needed gdal version on mac
>
> Please be much more specific. CRAN has macOS binaries of rgdal 1.5-8. Are
> you trying to install all the external software dependencies then rgdal
> from source? If this is difficult, rely on the CRAN macOS binaries, even
> though the PROJ and GDAL versions do not yet support WKT2 CRS
> representation. If you have installed PROJ and GDAL successfully, and PROJ
> < 6 && GDAL < 3, or PROJ >= 6 && GDAL >= 3, you should be OK. If you have
> the pointless PROJ >= 6 && GDAL < 3, you need the 1.5-9 tarball from
> R-forge, download locally and run R CMD INSTALL
> --configure-args="--with-proj_api=proj_api.h" rgdal_1.5-9.tar.gz.
>
> Please report back on specifics, or simply use CRAN binaries and report
> that.
>
> We are actively looking for ways to prevent macOS users getting trapped
> into trying to install from source when such a choice is not appropriate.
> We have no control over the provenance of the installed software as we
> have for Windows, thanks to Jeroen's rwinlib/gdal3 collection.
>
> Roger
>
> >
> > On Tue, Jun 2, 2020 at 4:04 PM Robin Lovelace <[hidden email]> wrote:
> >
> >> Confirmed: this worked perfectly for me.
> >>
> >> Video evidence that this is fixed:
> >> https://asciinema.org/a/zPQFbI6Q01oqUy1HRauCjLUXY
> >>
> >> Thanks Roger!
> >>
> >> Robin
> >>
> >> On Tue, Jun 2, 2020 at 4:00 PM Roger Bivand <[hidden email]>
> wrote:
> >>
> >>> Brian Ripley (thanks!) provided a patch for rgdal's configure.ac, and
> >> the
> >>> R-forge build and check processes now succeed, so please try to install
> >>> either downloading
> >>> http://download.r-forge.r-project.org/src/contrib/rgdal_1.5-9.tar.gz
> and
> >>> installing from the command line (after running R CMD check), or
> >>> install.packages("rgdal", repos="http://R-Forge.R-project.org").
> >>>
> >>> Remember that if GDAL is < 3, and PROJ >= 6, you need the configure
> >>> argument: --configure-args=--with-proj_api=proj_api.h.
> >>>
> >>> Roger
> >>>
> >>> On Sun, 31 May 2020, Roger Bivand wrote:
> >>>
> >>>> Fresh version of rgdal rev 995 on link and R-forge, with more tweaks
> to
> >>> the
> >>>> configure file. Please test.
> >>>>
> >>>> Roger
> >>>>
> >>>> On Fri, 29 May 2020, Roger Bivand wrote:
> >>>>
> >>>>>  I have put a tarball, built without vignettes with GDAL 2.2.4 and
> >> PROJ
> >>>>>  7.0.1 that passes CMD check (with warnings for missing vignettes)
> on:
> >>>>>
> >>>>>  http://spatial.nhh.no/R/rgdal/gdal_1.5-9.tar.gz
> >>>>>
> >>>>>  It involves code copying to provide duplicated functions in three
> >>>>>  settings:
> >>>>>
> >>>>>  PROJ < 6 & GDAL < 3
> >>>>> PROJ >  = 6 & GDAL >= 3
> >>>>> PROJ >  = 6 & GDAL < 3 (your homebrew case)
> >>>>>
> >>>>>  You need the configure argument:
> >>>>>  --configure-args=--with-proj_api=proj_api.h.
> >>>>>  Without the argument, you now get directed to use it.
> >>>>>
> >>>>>  For now I've dropped the configure tests for sqlite3, curl and tiff;
> >>> works
> >>>>>  for me but may not work for you.
> >>>>>
> >>>>>  Please make the output of:
> >>>>>
> >>>>>  R CMD check
> >>> --install-args="--configure-args=--with-proj_api=proj_api.h"
> >>>>>  rgdal_1.5-9.tar.gz
> >>>>>
> >>>>>  available ASAP. I count on an immediate response.
> >>>>>
> >>>>>  Roger
> >>>>>
> >>>>>  On Fri, 29 May 2020, Roger Bivand wrote:
> >>>>>
> >>>>>>   On Fri, 29 May 2020, Patrick Schratz wrote:
> >>>>>>
> >>>>>>>    The just released rgdal v1.5.8 requires gdal >= 3 when proj >= 6
> >> is
> >>>>>>>    present. It does not even install the package if this condition
> >> is
> >>> not
> >>>>>>>    met
> >>>>>>>    but errors during linking.
> >>>>>>
> >>>>>>   Please document with the output of 00install.out from R CMD check
> >>>>>>   rgdal_1.5-8.tar.gz, and pkg-config proj --modversion, gdalinfo
> >>> --version
> >>>>>>   with system details.
> >>>>>>
> >>>>>>   Yes, PROJ >= 6 makes no sense without GDAL >= 3; GDAL 3 makes no
> >>> sense
> >>>>>>   with PROJ < 6. GDAL 3 with PROJ < 6 uses the old proj_api.h API
> and
> >>> the
> >>>>>>   old metadata structure with Proj4 string degradation; GDAL < 3
> with
> >>> PROJ
> >>>>>>>   = 6 must use the deprecated API and the new metadata structure.
> >>>>>>
> >>>>>>>
> >>>>>>>    I could not find any sources in the web or in the announcement
> >>> post of
> >>>>>>>    v1.5.8 why this condition was enforced so strictly.
> >>>>>>>    Does it cause unwanted results?
> >>>>>>
> >>>>>>   Yes, definitely. GDAL 3 was released as 3, not 2.5, to stress the
> >>>>>>   complete
> >>>>>>   break between GDAL 2 (with PROJ <= 5) and GDAL >= 3 (with PROJ >=
> >> 6)
> >>>>>>   after
> >>>>>>   GDAL barnraising. So:
> >>>>>>
> >>>>>>             PROJ
> >>>>>>             < 6 >= 6
> >>>>>>   GDAL < 3   OK   NO
> >>>>>>   >   = 3   NO   OK
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>    If so, then there is a big issue since the “gdal2 and proj >= 6”
> >>>>>>>    combination has been used by many people in the past already.
> >>>>>>
> >>>>>>   No good precedent, just binary packagers protecting slow
> downstream
> >>>>>>   adaptation.
> >>>>>>
> >>>>>>>    I haven’t tracked down for how long this situation was on the
> >>> market
> >>>>>>>    already.
> >>>>>>>    Also the {sf} package is still installable with this combination
> >>> and I
> >>>>>>>    am not aware of any warnings/issues that came up due to this so
> >>> far.
> >>>>>>>
> >>>>>>
> >>>>>>   rgdal is a much older package and has much more older code that
> >> needs
> >>>>>>   this
> >>>>>>   protection. It is possible to accommodate the no-go areas, but I'd
> >>>>>>   really
> >>>>>>   value patches to R-forge, as I cannot check multiple PROJ/GDAL
> >>> version
> >>>>>>   combinations just because someone isn't up to speed.
> >>>>>>
> >>>>>>>    It further causes complete breakage for people relying on the
> >>> homebrew
> >>>>>>>    package manager on macOS since current versions are gdal v2.2.4
> >> and
> >>>>>>>    proj
> >>>>>>>    7.0.1.
> >>>>>>
> >>>>>>   They should never have gone beyond PROJ 5 if they are stuck on
> >> GDAL.
> >>>>>>   PROJ
> >>>>>>   7 is a very different animal.
> >>>>>>
> >>>>>>>    The update to gdal3 is blocked since months due to an
> >>> incompatibility
> >>>>>>>    with `liblas` (https://github.com/libLAS/libLAS/issues/164).
> >>> Reading
> >>>>>>>    the
> >>>>>>>    issue, it looks unclear when this issue will be resolved.
> >>>>>>>    The alternative osgeo4mac tap holds formula that is unstable and
> >>>>>>>    somewhat broken. One cannot link rgdal with the current gdal
> >> v3.0.1
> >>>>>>>    from
> >>>>>>>    osgeo4mac.
> >>>>>>>
> >>>>>>>    This incompatibility with homebrew-core formula leads to further
> >>>>>>>    issues
> >>>>>>>    for CI tests that rely on the spatial homebrew stack for macOS
> >>> builds.
> >>>>>>>
> >>>>>>>    All of this is really unfortunate and I am wondering if this was
> >> a
> >>>>>>>    known
> >>>>>>>    factor when introducing this condition in the new release.
> >>>>>>>    Unfortunately rgdal is not hosted on any Git* provider (there is
> >>> just
> >>>>>>>    the CRAN mirror with not much information about the recent
> >> changes)
> >>>>>>>    and
> >>>>>>>    it is unclear/untransparent to me if there is a continuous
> >>> integration
> >>>>>>>    setup at all for this package.
> >>>>>>
> >>>>>>   Do not require me to leave the excellent SVN service on R-forge.
> >>> Provide
> >>>>>>   patches there.
> >>>>>>
> >>>>>>>    I hope this is not another case in the series “I do not like
> >>> operating
> >>>>>>>    system XY and hence I do not test on it” which was seen recently
> >> in
> >>>>>>>    another package. That’s what CI is for and the responsibility of
> >>>>>>>    package authors.
> >>>>>>>
> >>>>>>
> >>>>>>   I run and develop on Fedora, I  have no access to OSX, and the
> >>> current
> >>>>>>   release was fully checked with ~ 900 revdeps repeatedly since
> >>> November
> >>>>>>   on
> >>>>>>   successive older and newer versions of PROJ and GDAL. I blocked
> new
> >>> PROJ
> >>>>>>   with old GDAL recently to simplify development. CI is only as good
> >> as
> >>>>>>   the
> >>>>>>   scripts, I am completely sure that repeated revdep testing and
> >>> reading
> >>>>>>   the
> >>>>>>   output is superior.
> >>>>>>
> >>>>>>>    Also one should be aware that not every rgdal user is member of
> >>> this
> >>>>>>>    mailing list (I guess not even 5%) and many people in production
> >>> will
> >>>>>>>    face this error during install, most of them without any clue
> >> what
> >>> to
> >>>>>>>    do
> >>>>>>>    or an idea why this was raised.
> >>>>>>>
> >>>>>>
> >>>>>>   There has been plenty of information given about the CRS changes.
> >>> rgdal
> >>>>>>   could simply have been abandoned by me, and all  those production
> >>>>>>   volunteers might have fixed things, but I never hear anything at
> >> all.
> >>>>>>
> >>>>>>>    In summary it would be great if
> >>>>>>>
> >>>>>>>    - There would be more detailed information on the introduction
> of
> >>> the
> >>>>>>>    new condition
> >>>>>>>    - Awareness of state-of-the-art platform related library
> versions
> >>>>>>>    before
> >>>>>>>    making such a release
> >>>>>>>    - Transparency/Existence of a cross-platform build process
> >>>>>>>    - If the incompatibility is just due to some features not being
> >>>>>>>    accessible with this configuration, I suggest to raise a warning
> >>>>>>>    during
> >>>>>>>    package load but not prevent the package from installing in the
> >>> first
> >>>>>>>    place
> >>>>>>>    - A version-based NEWS file would be available. Currently one
> >> only
> >>>>>>>    sees
> >>>>>>>    a revision-based Changelog:
> >>>>>>>    https://cran.r-project.org/web/packages/rgdal/ChangeLog
> >>>>>>>
> >>>>>>>    I am well aware that I am indirectly criticising (hopefully in a
> >>>>>>>    constructive way) the transparency and release process here, for
> >>> which
> >>>>>>>    I
> >>>>>>>    will probably get a rough response.
> >>>>>>>    However, if that leads to more robustness and transparency in
> >>> future
> >>>>>>>    release, I am fine with this.
> >>>>>>>
> >>>>>>>    A quick patch release resolving the breakage would be highly
> >>>>>>>    appreciated.
> >>>>>>
> >>>>>>   Only with community imput. what you ask is not needed, just extra
> >>>>>>   complexity. Please provide patches, or accept my invitation to
> join
> >>> the
> >>>>>>   R-forge project and commit your fixes directly. I can see how to
> do
> >>> it,
> >>>>>>   but I don't think it makes sense, and your messsage has not
> >> motivated
> >>>>>>   me,
> >>>>>>   to be honest. I'm prioritizing working with CRAN to iron out
> >> reverse
> >>>>>>   dependency problems.
> >>>>>>
> >>>>>>   Roger
> >>>>>>
> >>>>>>>
> >>>>>>>    Best, Patrick
> >>>>>>>     [[alternative HTML version deleted]]
> >>>>>>>
> >>>>>>>    _______________________________________________
> >>>>>>>    R-sig-Geo mailing list
> >>>>>>>    [hidden email]
> >>>>>>>    https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Roger Bivand
> >>> Department of Economics, Norwegian School of Economics,
> >>> Helleveien 30, N-5045 Bergen, Norway.
> >>> voice: +47 55 95 93 55; e-mail: [hidden email]
> >>> https://orcid.org/0000-0003-2392-6140
> >>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> >>> _______________________________________________
> >>> R-sig-Geo mailing list
> >>> [hidden email]
> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>>
> >>
> >>         [[alternative HTML version deleted]]
> >>
> >> _______________________________________________
> >> R-sig-Geo mailing list
> >> [hidden email]
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
>
> --
> Roger Bivand
> Department of Economics, Norwegian School of Economics,
> Helleveien 30, N-5045 Bergen, Norway.
> voice: +47 55 95 93 55; e-mail: [hidden email]
> https://orcid.org/0000-0003-2392-6140
> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: Package lwgeom (0.2-4) fails to install on Shinyapps.io with an "upgrade GEOS to 3.6.0 or later" error

Wed, 06/03/2020 - 12:21
The documentation at
https://docs.rstudio.com/shinyapps.io/getting-started.html#deploying-applications
will help you figure it out. From a quick glance it seems you can
either downgrade your local version of lwgeom or work to get the
system dependencies installed as instructed at
https://docs.rstudio.com/shinyapps.io/getting-started.html#system-packages

Best,
Ista

On Wed, Jun 3, 2020 at 12:00 PM El Mechry El Koudouss
<[hidden email]> wrote:
>
> Dear all,
> I am building a Shiny app that works fine on my laptop (Windows 10).
> However, it fails to deploy on Shinyapps.io because the package lwgeom
> (0.2-4) fails to install on Shinyapps.io. I have the sf  package installed
> and loaded to the library on my laptop and it links to GEOS 3.8.0. I also
> have the  lwgeom package installed and loaded to the library in my laptop
> and it also links to GOES 3.8.0. Yes the error I get is:
> error: upgrade GEOS to 3.6.0 or later
> ERROR: configuration failed for package ‘lwgeom’
> The full message I get up on trying to deploy my app is pasted below. For
> more information, I am using the following packages and the following shiny
> functions. I am hoping someone here can point me in the right direction.
> Thanks.
>
> library(shiny)
> library(shinythemes)
> library(plotly)
> library(tidyverse)
> library(sf)
> library(leaflet)
> library(tmap)
> library(lubridate)
> renderLeaflet()
> tm_shape()
> tm_polygons()
> tmap_leaflet()
>
> Full error message is below:
> Building R package: lwgeom (0.2-4)
> /mnt/packages/build /mnt
> * installing to library ‘/opt/R/4.0.0/lib/R/library’
> * installing *source* package ‘lwgeom’ ...
> ** package ‘lwgeom’ successfully unpacked and MD5 sums checked
> ** using staged installation
> configure: CC: gcc
> configure: CXX: g++ -std=gnu++11
> configure: pkg-config proj exists, will use it
> configure: PROJ: 4.9.2
> checking for pj_init_plus in -lproj... yes
> checking PROJ: epsg found and readable... yes
> configure: POSTGIS_PROJ_VERSION: 49
> checking for geos-config... /usr/bin/geos-config
> checking geos-config usability... yes
> configure: GEOS: 3.5.1
> checking GEOS version >= 3.6.0... no
> configure: error: upgrade GEOS to 3.6.0 or later
> ERROR: configuration failed for package ‘lwgeom’
> * removing ‘/opt/R/4.0.0/lib/R/library/lwgeom’
> ################################# End Task Log
> #################################
> Error: Unhandled Exception: Child Task 741259659 failed: Error building
> image: Error building lwgeom (0.2-4). Build exited with non-zero status: 1
> Execution halted
>
> --
> El Mechry, El Koudouss (Meshry)
> Graduate Research Assistant
> Center for International Policy Studies
> Fordham University
> Department of Economics
> Website: www.meshry.com
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: Package lwgeom (0.2-4) fails to install on Shinyapps.io with an "upgrade GEOS to 3.6.0 or later" error

Wed, 06/03/2020 - 11:52
R package lwgeom had to upgrade to a newer version of liblwgeom (now a
subdirectory in the PostGIS source tree, really) because of the new PROJ
versions and deprecation of the old PROJ interface; PostGIS requires the
newer GEOS version.

I have no control over versions of GEOS that are run on the shinyapps.io
machines, and don't know which OS at which version they run. I also have
no clue how one could debug this issue.

One could try to patch the install file,
https://github.com/rstudio/shinyapps-package-dependencies/blob/master/packages/lwgeom/install
such that it installs a newer GEOS (maybe binary from another ppa,
otherwise from source).

On 6/3/20 6:33 PM, Rich Shepard wrote:
> On Wed, 3 Jun 2020, Erin Hodgess wrote:
>
>> Could this possibly be due to the upgrade for GDAL and PROJ libraries,
>> please?
>
> Erin,
>
> I learned a while ago that when any of proj, geos, and gdal are upgraded I
> need to rebuid tools higher in that chain. If you have a new geos version
> then a new gdal build is in order.
>
> If this does not address your issue please excuse my responding.
>
> Regards,
>
> Rich
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo--
Edzer Pebesma
Institute for Geoinformatics
Heisenbergstrasse 2, 48149 Muenster, Germany
Phone: +49 251 8333081

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

pEpkey.asc (3K) Download Attachment

Re: Package lwgeom (0.2-4) fails to install on Shinyapps.io with an "upgrade GEOS to 3.6.0 or later" error

Wed, 06/03/2020 - 11:33
On Wed, 3 Jun 2020, Erin Hodgess wrote:

> Could this possibly be due to the upgrade for GDAL and PROJ libraries,
> please?

Erin,

I learned a while ago that when any of proj, geos, and gdal are upgraded I
need to rebuid tools higher in that chain. If you have a new geos version
then a new gdal build is in order.

If this does not address your issue please excuse my responding.

Regards,

Rich

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: Package lwgeom (0.2-4) fails to install on Shinyapps.io with an "upgrade GEOS to 3.6.0 or later" error

Wed, 06/03/2020 - 11:03
Hello!

Could this possibly be due to the upgrade for GDAL and PROJ libraries,
please?

That might be something on the Shinyapps.io side.  Just a thought.

Thanks,
Erin

On Wed, Jun 3, 2020 at 10:00 AM El Mechry El Koudouss <
[hidden email]> wrote:

> Dear all,
> I am building a Shiny app that works fine on my laptop (Windows 10).
> However, it fails to deploy on Shinyapps.io because the package lwgeom
> (0.2-4) fails to install on Shinyapps.io. I have the sf  package installed
> and loaded to the library on my laptop and it links to GEOS 3.8.0. I also
> have the  lwgeom package installed and loaded to the library in my laptop
> and it also links to GOES 3.8.0. Yes the error I get is:
> error: upgrade GEOS to 3.6.0 or later
> ERROR: configuration failed for package ‘lwgeom’
> The full message I get up on trying to deploy my app is pasted below. For
> more information, I am using the following packages and the following shiny
> functions. I am hoping someone here can point me in the right direction.
> Thanks.
>
> library(shiny)
> library(shinythemes)
> library(plotly)
> library(tidyverse)
> library(sf)
> library(leaflet)
> library(tmap)
> library(lubridate)
> renderLeaflet()
> tm_shape()
> tm_polygons()
> tmap_leaflet()
>
> Full error message is below:
> Building R package: lwgeom (0.2-4)
> /mnt/packages/build /mnt
> * installing to library ‘/opt/R/4.0.0/lib/R/library’
> * installing *source* package ‘lwgeom’ ...
> ** package ‘lwgeom’ successfully unpacked and MD5 sums checked
> ** using staged installation
> configure: CC: gcc
> configure: CXX: g++ -std=gnu++11
> configure: pkg-config proj exists, will use it
> configure: PROJ: 4.9.2
> checking for pj_init_plus in -lproj... yes
> checking PROJ: epsg found and readable... yes
> configure: POSTGIS_PROJ_VERSION: 49
> checking for geos-config... /usr/bin/geos-config
> checking geos-config usability... yes
> configure: GEOS: 3.5.1
> checking GEOS version >= 3.6.0... no
> configure: error: upgrade GEOS to 3.6.0 or later
> ERROR: configuration failed for package ‘lwgeom’
> * removing ‘/opt/R/4.0.0/lib/R/library/lwgeom’
> ################################# End Task Log
> #################################
> Error: Unhandled Exception: Child Task 741259659 failed: Error building
> image: Error building lwgeom (0.2-4). Build exited with non-zero status: 1
> Execution halted
>
> --
> El Mechry, El Koudouss (Meshry)
> Graduate Research Assistant
> Center for International Policy Studies
> Fordham University
> Department of Economics
> Website: www.meshry.com
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> --
Erin Hodgess, PhD
mailto: [hidden email]

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Package lwgeom (0.2-4) fails to install on Shinyapps.io with an "upgrade GEOS to 3.6.0 or later" error

Wed, 06/03/2020 - 10:57
Dear all,
I am building a Shiny app that works fine on my laptop (Windows 10).
However, it fails to deploy on Shinyapps.io because the package lwgeom
(0.2-4) fails to install on Shinyapps.io. I have the sf  package installed
and loaded to the library on my laptop and it links to GEOS 3.8.0. I also
have the  lwgeom package installed and loaded to the library in my laptop
and it also links to GOES 3.8.0. Yes the error I get is:
error: upgrade GEOS to 3.6.0 or later
ERROR: configuration failed for package ‘lwgeom’
The full message I get up on trying to deploy my app is pasted below. For
more information, I am using the following packages and the following shiny
functions. I am hoping someone here can point me in the right direction.
Thanks.

library(shiny)
library(shinythemes)
library(plotly)
library(tidyverse)
library(sf)
library(leaflet)
library(tmap)
library(lubridate)
renderLeaflet()
tm_shape()
tm_polygons()
tmap_leaflet()

Full error message is below:
Building R package: lwgeom (0.2-4)
/mnt/packages/build /mnt
* installing to library ‘/opt/R/4.0.0/lib/R/library’
* installing *source* package ‘lwgeom’ ...
** package ‘lwgeom’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: gcc
configure: CXX: g++ -std=gnu++11
configure: pkg-config proj exists, will use it
configure: PROJ: 4.9.2
checking for pj_init_plus in -lproj... yes
checking PROJ: epsg found and readable... yes
configure: POSTGIS_PROJ_VERSION: 49
checking for geos-config... /usr/bin/geos-config
checking geos-config usability... yes
configure: GEOS: 3.5.1
checking GEOS version >= 3.6.0... no
configure: error: upgrade GEOS to 3.6.0 or later
ERROR: configuration failed for package ‘lwgeom’
* removing ‘/opt/R/4.0.0/lib/R/library/lwgeom’
################################# End Task Log
#################################
Error: Unhandled Exception: Child Task 741259659 failed: Error building
image: Error building lwgeom (0.2-4). Build exited with non-zero status: 1
Execution halted

--
El Mechry, El Koudouss (Meshry)
Graduate Research Assistant
Center for International Policy Studies
Fordham University
Department of Economics
Website: www.meshry.com

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: rgdal: About the new gdal3 and proj >=6 condition

Wed, 06/03/2020 - 10:30
On Wed, 3 Jun 2020, Abdoulaye Sarr wrote:

> Have a problem with needed gdal version on mac

Please be much more specific. CRAN has macOS binaries of rgdal 1.5-8. Are
you trying to install all the external software dependencies then rgdal
from source? If this is difficult, rely on the CRAN macOS binaries, even
though the PROJ and GDAL versions do not yet support WKT2 CRS
representation. If you have installed PROJ and GDAL successfully, and PROJ
< 6 && GDAL < 3, or PROJ >= 6 && GDAL >= 3, you should be OK. If you have
the pointless PROJ >= 6 && GDAL < 3, you need the 1.5-9 tarball from
R-forge, download locally and run R CMD INSTALL
--configure-args="--with-proj_api=proj_api.h" rgdal_1.5-9.tar.gz.

Please report back on specifics, or simply use CRAN binaries and report
that.

We are actively looking for ways to prevent macOS users getting trapped
into trying to install from source when such a choice is not appropriate.
We have no control over the provenance of the installed software as we
have for Windows, thanks to Jeroen's rwinlib/gdal3 collection.

Roger

>
> On Tue, Jun 2, 2020 at 4:04 PM Robin Lovelace <[hidden email]> wrote:
>
>> Confirmed: this worked perfectly for me.
>>
>> Video evidence that this is fixed:
>> https://asciinema.org/a/zPQFbI6Q01oqUy1HRauCjLUXY
>>
>> Thanks Roger!
>>
>> Robin
>>
>> On Tue, Jun 2, 2020 at 4:00 PM Roger Bivand <[hidden email]> wrote:
>>
>>> Brian Ripley (thanks!) provided a patch for rgdal's configure.ac, and
>> the
>>> R-forge build and check processes now succeed, so please try to install
>>> either downloading
>>> http://download.r-forge.r-project.org/src/contrib/rgdal_1.5-9.tar.gz and
>>> installing from the command line (after running R CMD check), or
>>> install.packages("rgdal", repos="http://R-Forge.R-project.org").
>>>
>>> Remember that if GDAL is < 3, and PROJ >= 6, you need the configure
>>> argument: --configure-args=--with-proj_api=proj_api.h.
>>>
>>> Roger
>>>
>>> On Sun, 31 May 2020, Roger Bivand wrote:
>>>
>>>> Fresh version of rgdal rev 995 on link and R-forge, with more tweaks to
>>> the
>>>> configure file. Please test.
>>>>
>>>> Roger
>>>>
>>>> On Fri, 29 May 2020, Roger Bivand wrote:
>>>>
>>>>>  I have put a tarball, built without vignettes with GDAL 2.2.4 and
>> PROJ
>>>>>  7.0.1 that passes CMD check (with warnings for missing vignettes) on:
>>>>>
>>>>>  http://spatial.nhh.no/R/rgdal/gdal_1.5-9.tar.gz
>>>>>
>>>>>  It involves code copying to provide duplicated functions in three
>>>>>  settings:
>>>>>
>>>>>  PROJ < 6 & GDAL < 3
>>>>> PROJ >  = 6 & GDAL >= 3
>>>>> PROJ >  = 6 & GDAL < 3 (your homebrew case)
>>>>>
>>>>>  You need the configure argument:
>>>>>  --configure-args=--with-proj_api=proj_api.h.
>>>>>  Without the argument, you now get directed to use it.
>>>>>
>>>>>  For now I've dropped the configure tests for sqlite3, curl and tiff;
>>> works
>>>>>  for me but may not work for you.
>>>>>
>>>>>  Please make the output of:
>>>>>
>>>>>  R CMD check
>>> --install-args="--configure-args=--with-proj_api=proj_api.h"
>>>>>  rgdal_1.5-9.tar.gz
>>>>>
>>>>>  available ASAP. I count on an immediate response.
>>>>>
>>>>>  Roger
>>>>>
>>>>>  On Fri, 29 May 2020, Roger Bivand wrote:
>>>>>
>>>>>>   On Fri, 29 May 2020, Patrick Schratz wrote:
>>>>>>
>>>>>>>    The just released rgdal v1.5.8 requires gdal >= 3 when proj >= 6
>> is
>>>>>>>    present. It does not even install the package if this condition
>> is
>>> not
>>>>>>>    met
>>>>>>>    but errors during linking.
>>>>>>
>>>>>>   Please document with the output of 00install.out from R CMD check
>>>>>>   rgdal_1.5-8.tar.gz, and pkg-config proj --modversion, gdalinfo
>>> --version
>>>>>>   with system details.
>>>>>>
>>>>>>   Yes, PROJ >= 6 makes no sense without GDAL >= 3; GDAL 3 makes no
>>> sense
>>>>>>   with PROJ < 6. GDAL 3 with PROJ < 6 uses the old proj_api.h API and
>>> the
>>>>>>   old metadata structure with Proj4 string degradation; GDAL < 3 with
>>> PROJ
>>>>>>>   = 6 must use the deprecated API and the new metadata structure.
>>>>>>
>>>>>>>
>>>>>>>    I could not find any sources in the web or in the announcement
>>> post of
>>>>>>>    v1.5.8 why this condition was enforced so strictly.
>>>>>>>    Does it cause unwanted results?
>>>>>>
>>>>>>   Yes, definitely. GDAL 3 was released as 3, not 2.5, to stress the
>>>>>>   complete
>>>>>>   break between GDAL 2 (with PROJ <= 5) and GDAL >= 3 (with PROJ >=
>> 6)
>>>>>>   after
>>>>>>   GDAL barnraising. So:
>>>>>>
>>>>>>             PROJ
>>>>>>             < 6 >= 6
>>>>>>   GDAL < 3   OK   NO
>>>>>>   >   = 3   NO   OK
>>>>>>
>>>>>>
>>>>>>
>>>>>>>    If so, then there is a big issue since the “gdal2 and proj >= 6”
>>>>>>>    combination has been used by many people in the past already.
>>>>>>
>>>>>>   No good precedent, just binary packagers protecting slow downstream
>>>>>>   adaptation.
>>>>>>
>>>>>>>    I haven’t tracked down for how long this situation was on the
>>> market
>>>>>>>    already.
>>>>>>>    Also the {sf} package is still installable with this combination
>>> and I
>>>>>>>    am not aware of any warnings/issues that came up due to this so
>>> far.
>>>>>>>
>>>>>>
>>>>>>   rgdal is a much older package and has much more older code that
>> needs
>>>>>>   this
>>>>>>   protection. It is possible to accommodate the no-go areas, but I'd
>>>>>>   really
>>>>>>   value patches to R-forge, as I cannot check multiple PROJ/GDAL
>>> version
>>>>>>   combinations just because someone isn't up to speed.
>>>>>>
>>>>>>>    It further causes complete breakage for people relying on the
>>> homebrew
>>>>>>>    package manager on macOS since current versions are gdal v2.2.4
>> and
>>>>>>>    proj
>>>>>>>    7.0.1.
>>>>>>
>>>>>>   They should never have gone beyond PROJ 5 if they are stuck on
>> GDAL.
>>>>>>   PROJ
>>>>>>   7 is a very different animal.
>>>>>>
>>>>>>>    The update to gdal3 is blocked since months due to an
>>> incompatibility
>>>>>>>    with `liblas` (https://github.com/libLAS/libLAS/issues/164).
>>> Reading
>>>>>>>    the
>>>>>>>    issue, it looks unclear when this issue will be resolved.
>>>>>>>    The alternative osgeo4mac tap holds formula that is unstable and
>>>>>>>    somewhat broken. One cannot link rgdal with the current gdal
>> v3.0.1
>>>>>>>    from
>>>>>>>    osgeo4mac.
>>>>>>>
>>>>>>>    This incompatibility with homebrew-core formula leads to further
>>>>>>>    issues
>>>>>>>    for CI tests that rely on the spatial homebrew stack for macOS
>>> builds.
>>>>>>>
>>>>>>>    All of this is really unfortunate and I am wondering if this was
>> a
>>>>>>>    known
>>>>>>>    factor when introducing this condition in the new release.
>>>>>>>    Unfortunately rgdal is not hosted on any Git* provider (there is
>>> just
>>>>>>>    the CRAN mirror with not much information about the recent
>> changes)
>>>>>>>    and
>>>>>>>    it is unclear/untransparent to me if there is a continuous
>>> integration
>>>>>>>    setup at all for this package.
>>>>>>
>>>>>>   Do not require me to leave the excellent SVN service on R-forge.
>>> Provide
>>>>>>   patches there.
>>>>>>
>>>>>>>    I hope this is not another case in the series “I do not like
>>> operating
>>>>>>>    system XY and hence I do not test on it” which was seen recently
>> in
>>>>>>>    another package. That’s what CI is for and the responsibility of
>>>>>>>    package authors.
>>>>>>>
>>>>>>
>>>>>>   I run and develop on Fedora, I  have no access to OSX, and the
>>> current
>>>>>>   release was fully checked with ~ 900 revdeps repeatedly since
>>> November
>>>>>>   on
>>>>>>   successive older and newer versions of PROJ and GDAL. I blocked new
>>> PROJ
>>>>>>   with old GDAL recently to simplify development. CI is only as good
>> as
>>>>>>   the
>>>>>>   scripts, I am completely sure that repeated revdep testing and
>>> reading
>>>>>>   the
>>>>>>   output is superior.
>>>>>>
>>>>>>>    Also one should be aware that not every rgdal user is member of
>>> this
>>>>>>>    mailing list (I guess not even 5%) and many people in production
>>> will
>>>>>>>    face this error during install, most of them without any clue
>> what
>>> to
>>>>>>>    do
>>>>>>>    or an idea why this was raised.
>>>>>>>
>>>>>>
>>>>>>   There has been plenty of information given about the CRS changes.
>>> rgdal
>>>>>>   could simply have been abandoned by me, and all  those production
>>>>>>   volunteers might have fixed things, but I never hear anything at
>> all.
>>>>>>
>>>>>>>    In summary it would be great if
>>>>>>>
>>>>>>>    - There would be more detailed information on the introduction of
>>> the
>>>>>>>    new condition
>>>>>>>    - Awareness of state-of-the-art platform related library versions
>>>>>>>    before
>>>>>>>    making such a release
>>>>>>>    - Transparency/Existence of a cross-platform build process
>>>>>>>    - If the incompatibility is just due to some features not being
>>>>>>>    accessible with this configuration, I suggest to raise a warning
>>>>>>>    during
>>>>>>>    package load but not prevent the package from installing in the
>>> first
>>>>>>>    place
>>>>>>>    - A version-based NEWS file would be available. Currently one
>> only
>>>>>>>    sees
>>>>>>>    a revision-based Changelog:
>>>>>>>    https://cran.r-project.org/web/packages/rgdal/ChangeLog
>>>>>>>
>>>>>>>    I am well aware that I am indirectly criticising (hopefully in a
>>>>>>>    constructive way) the transparency and release process here, for
>>> which
>>>>>>>    I
>>>>>>>    will probably get a rough response.
>>>>>>>    However, if that leads to more robustness and transparency in
>>> future
>>>>>>>    release, I am fine with this.
>>>>>>>
>>>>>>>    A quick patch release resolving the breakage would be highly
>>>>>>>    appreciated.
>>>>>>
>>>>>>   Only with community imput. what you ask is not needed, just extra
>>>>>>   complexity. Please provide patches, or accept my invitation to join
>>> the
>>>>>>   R-forge project and commit your fixes directly. I can see how to do
>>> it,
>>>>>>   but I don't think it makes sense, and your messsage has not
>> motivated
>>>>>>   me,
>>>>>>   to be honest. I'm prioritizing working with CRAN to iron out
>> reverse
>>>>>>   dependency problems.
>>>>>>
>>>>>>   Roger
>>>>>>
>>>>>>>
>>>>>>>    Best, Patrick
>>>>>>>     [[alternative HTML version deleted]]
>>>>>>>
>>>>>>>    _______________________________________________
>>>>>>>    R-sig-Geo mailing list
>>>>>>>    [hidden email]
>>>>>>>    https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, Norwegian School of Economics,
>>> Helleveien 30, N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>> https://orcid.org/0000-0003-2392-6140
>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>> _______________________________________________
>>> R-sig-Geo mailing list
>>> [hidden email]
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> [hidden email]
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> --
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway

Re: rgdal: About the new gdal3 and proj >=6 condition

Wed, 06/03/2020 - 08:38
Have a problem with needed gdal version on mac

On Tue, Jun 2, 2020 at 4:04 PM Robin Lovelace <[hidden email]> wrote:

> Confirmed: this worked perfectly for me.
>
> Video evidence that this is fixed:
> https://asciinema.org/a/zPQFbI6Q01oqUy1HRauCjLUXY
>
> Thanks Roger!
>
> Robin
>
> On Tue, Jun 2, 2020 at 4:00 PM Roger Bivand <[hidden email]> wrote:
>
> > Brian Ripley (thanks!) provided a patch for rgdal's configure.ac, and
> the
> > R-forge build and check processes now succeed, so please try to install
> > either downloading
> > http://download.r-forge.r-project.org/src/contrib/rgdal_1.5-9.tar.gz and
> > installing from the command line (after running R CMD check), or
> > install.packages("rgdal", repos="http://R-Forge.R-project.org").
> >
> > Remember that if GDAL is < 3, and PROJ >= 6, you need the configure
> > argument: --configure-args=--with-proj_api=proj_api.h.
> >
> > Roger
> >
> > On Sun, 31 May 2020, Roger Bivand wrote:
> >
> > > Fresh version of rgdal rev 995 on link and R-forge, with more tweaks to
> > the
> > > configure file. Please test.
> > >
> > > Roger
> > >
> > > On Fri, 29 May 2020, Roger Bivand wrote:
> > >
> > >>  I have put a tarball, built without vignettes with GDAL 2.2.4 and
> PROJ
> > >>  7.0.1 that passes CMD check (with warnings for missing vignettes) on:
> > >>
> > >>  http://spatial.nhh.no/R/rgdal/gdal_1.5-9.tar.gz
> > >>
> > >>  It involves code copying to provide duplicated functions in three
> > >>  settings:
> > >>
> > >>  PROJ < 6 & GDAL < 3
> > >> PROJ >  = 6 & GDAL >= 3
> > >> PROJ >  = 6 & GDAL < 3 (your homebrew case)
> > >>
> > >>  You need the configure argument:
> > >>  --configure-args=--with-proj_api=proj_api.h.
> > >>  Without the argument, you now get directed to use it.
> > >>
> > >>  For now I've dropped the configure tests for sqlite3, curl and tiff;
> > works
> > >>  for me but may not work for you.
> > >>
> > >>  Please make the output of:
> > >>
> > >>  R CMD check
> > --install-args="--configure-args=--with-proj_api=proj_api.h"
> > >>  rgdal_1.5-9.tar.gz
> > >>
> > >>  available ASAP. I count on an immediate response.
> > >>
> > >>  Roger
> > >>
> > >>  On Fri, 29 May 2020, Roger Bivand wrote:
> > >>
> > >>>   On Fri, 29 May 2020, Patrick Schratz wrote:
> > >>>
> > >>>>    The just released rgdal v1.5.8 requires gdal >= 3 when proj >= 6
> is
> > >>>>    present. It does not even install the package if this condition
> is
> > not
> > >>>>    met
> > >>>>    but errors during linking.
> > >>>
> > >>>   Please document with the output of 00install.out from R CMD check
> > >>>   rgdal_1.5-8.tar.gz, and pkg-config proj --modversion, gdalinfo
> > --version
> > >>>   with system details.
> > >>>
> > >>>   Yes, PROJ >= 6 makes no sense without GDAL >= 3; GDAL 3 makes no
> > sense
> > >>>   with PROJ < 6. GDAL 3 with PROJ < 6 uses the old proj_api.h API and
> > the
> > >>>   old metadata structure with Proj4 string degradation; GDAL < 3 with
> > PROJ
> > >>>>   = 6 must use the deprecated API and the new metadata structure.
> > >>>
> > >>>>
> > >>>>    I could not find any sources in the web or in the announcement
> > post of
> > >>>>    v1.5.8 why this condition was enforced so strictly.
> > >>>>    Does it cause unwanted results?
> > >>>
> > >>>   Yes, definitely. GDAL 3 was released as 3, not 2.5, to stress the
> > >>>   complete
> > >>>   break between GDAL 2 (with PROJ <= 5) and GDAL >= 3 (with PROJ >=
> 6)
> > >>>   after
> > >>>   GDAL barnraising. So:
> > >>>
> > >>>             PROJ
> > >>>             < 6 >= 6
> > >>>   GDAL < 3   OK   NO
> > >>>    >   = 3   NO   OK
> > >>>
> > >>>
> > >>>
> > >>>>    If so, then there is a big issue since the “gdal2 and proj >= 6”
> > >>>>    combination has been used by many people in the past already.
> > >>>
> > >>>   No good precedent, just binary packagers protecting slow downstream
> > >>>   adaptation.
> > >>>
> > >>>>    I haven’t tracked down for how long this situation was on the
> > market
> > >>>>    already.
> > >>>>    Also the {sf} package is still installable with this combination
> > and I
> > >>>>    am not aware of any warnings/issues that came up due to this so
> > far.
> > >>>>
> > >>>
> > >>>   rgdal is a much older package and has much more older code that
> needs
> > >>>   this
> > >>>   protection. It is possible to accommodate the no-go areas, but I'd
> > >>>   really
> > >>>   value patches to R-forge, as I cannot check multiple PROJ/GDAL
> > version
> > >>>   combinations just because someone isn't up to speed.
> > >>>
> > >>>>    It further causes complete breakage for people relying on the
> > homebrew
> > >>>>    package manager on macOS since current versions are gdal v2.2.4
> and
> > >>>>    proj
> > >>>>    7.0.1.
> > >>>
> > >>>   They should never have gone beyond PROJ 5 if they are stuck on
> GDAL.
> > >>>   PROJ
> > >>>   7 is a very different animal.
> > >>>
> > >>>>    The update to gdal3 is blocked since months due to an
> > incompatibility
> > >>>>    with `liblas` (https://github.com/libLAS/libLAS/issues/164).
> > Reading
> > >>>>    the
> > >>>>    issue, it looks unclear when this issue will be resolved.
> > >>>>    The alternative osgeo4mac tap holds formula that is unstable and
> > >>>>    somewhat broken. One cannot link rgdal with the current gdal
> v3.0.1
> > >>>>    from
> > >>>>    osgeo4mac.
> > >>>>
> > >>>>    This incompatibility with homebrew-core formula leads to further
> > >>>>    issues
> > >>>>    for CI tests that rely on the spatial homebrew stack for macOS
> > builds.
> > >>>>
> > >>>>    All of this is really unfortunate and I am wondering if this was
> a
> > >>>>    known
> > >>>>    factor when introducing this condition in the new release.
> > >>>>    Unfortunately rgdal is not hosted on any Git* provider (there is
> > just
> > >>>>    the CRAN mirror with not much information about the recent
> changes)
> > >>>>    and
> > >>>>    it is unclear/untransparent to me if there is a continuous
> > integration
> > >>>>    setup at all for this package.
> > >>>
> > >>>   Do not require me to leave the excellent SVN service on R-forge.
> > Provide
> > >>>   patches there.
> > >>>
> > >>>>    I hope this is not another case in the series “I do not like
> > operating
> > >>>>    system XY and hence I do not test on it” which was seen recently
> in
> > >>>>    another package. That’s what CI is for and the responsibility of
> > >>>>    package authors.
> > >>>>
> > >>>
> > >>>   I run and develop on Fedora, I  have no access to OSX, and the
> > current
> > >>>   release was fully checked with ~ 900 revdeps repeatedly since
> > November
> > >>>   on
> > >>>   successive older and newer versions of PROJ and GDAL. I blocked new
> > PROJ
> > >>>   with old GDAL recently to simplify development. CI is only as good
> as
> > >>>   the
> > >>>   scripts, I am completely sure that repeated revdep testing and
> > reading
> > >>>   the
> > >>>   output is superior.
> > >>>
> > >>>>    Also one should be aware that not every rgdal user is member of
> > this
> > >>>>    mailing list (I guess not even 5%) and many people in production
> > will
> > >>>>    face this error during install, most of them without any clue
> what
> > to
> > >>>>    do
> > >>>>    or an idea why this was raised.
> > >>>>
> > >>>
> > >>>   There has been plenty of information given about the CRS changes.
> > rgdal
> > >>>   could simply have been abandoned by me, and all  those production
> > >>>   volunteers might have fixed things, but I never hear anything at
> all.
> > >>>
> > >>>>    In summary it would be great if
> > >>>>
> > >>>>    - There would be more detailed information on the introduction of
> > the
> > >>>>    new condition
> > >>>>    - Awareness of state-of-the-art platform related library versions
> > >>>>    before
> > >>>>    making such a release
> > >>>>    - Transparency/Existence of a cross-platform build process
> > >>>>    - If the incompatibility is just due to some features not being
> > >>>>    accessible with this configuration, I suggest to raise a warning
> > >>>>    during
> > >>>>    package load but not prevent the package from installing in the
> > first
> > >>>>    place
> > >>>>    - A version-based NEWS file would be available. Currently one
> only
> > >>>>    sees
> > >>>>    a revision-based Changelog:
> > >>>>    https://cran.r-project.org/web/packages/rgdal/ChangeLog
> > >>>>
> > >>>>    I am well aware that I am indirectly criticising (hopefully in a
> > >>>>    constructive way) the transparency and release process here, for
> > which
> > >>>>    I
> > >>>>    will probably get a rough response.
> > >>>>    However, if that leads to more robustness and transparency in
> > future
> > >>>>    release, I am fine with this.
> > >>>>
> > >>>>    A quick patch release resolving the breakage would be highly
> > >>>>    appreciated.
> > >>>
> > >>>   Only with community imput. what you ask is not needed, just extra
> > >>>   complexity. Please provide patches, or accept my invitation to join
> > the
> > >>>   R-forge project and commit your fixes directly. I can see how to do
> > it,
> > >>>   but I don't think it makes sense, and your messsage has not
> motivated
> > >>>   me,
> > >>>   to be honest. I'm prioritizing working with CRAN to iron out
> reverse
> > >>>   dependency problems.
> > >>>
> > >>>   Roger
> > >>>
> > >>>>
> > >>>>    Best, Patrick
> > >>>>     [[alternative HTML version deleted]]
> > >>>>
> > >>>>    _______________________________________________
> > >>>>    R-sig-Geo mailing list
> > >>>>    [hidden email]
> > >>>>    https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> > >>>>
> > >>>
> > >>>
> > >>
> > >>
> > >
> > >
> >
> > --
> > Roger Bivand
> > Department of Economics, Norwegian School of Economics,
> > Helleveien 30, N-5045 Bergen, Norway.
> > voice: +47 55 95 93 55; e-mail: [hidden email]
> > https://orcid.org/0000-0003-2392-6140
> > https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
> > _______________________________________________
> > R-sig-Geo mailing list
> > [hidden email]
> > https://stat.ethz.ch/mailman/listinfo/r-sig-geo
> >
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> [hidden email]
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Re: How to efficiently generate data of neighboring points

Wed, 06/03/2020 - 08:18
On Wed, 3 Jun 2020, Lom Navanyo wrote:

> I had the errors with rtree using R 3.6.3. I have since changed to R 4.0.0
> but I got the same error.
>
> And  yes, for Roger's example, I have the objects wd1, ... wd4, all with
> length 101. I think my difficulty is my inability to output the list
> detailing the point IDs t50_fid.

library(spData)
library(sf)
projdata<-st_transform(nz_height, 32759)
pts <- st_coordinates(projdata)
library(spdep)
bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
bds <- c(0, bufferR)
wd1 <- dnearneigh(pts, bds[1], bds[2])
wd2 <- dnearneigh(pts, bds[2], bds[3])
wd3 <- dnearneigh(pts, bds[3], bds[4])
wd4 <- dnearneigh(pts, bds[4], bds[5])
sn_band1 <- listw2sn(nb2listw(wd1, style="B", zero.policy=TRUE))
sn_band1$band <- paste(attr(wd1, "distances"), collapse="-")
sn_band2 <- listw2sn(nb2listw(wd2, style="B", zero.policy=TRUE))
sn_band2$band <- paste(attr(wd2, "distances"), collapse="-")
sn_band3 <- listw2sn(nb2listw(wd3, style="B", zero.policy=TRUE))
sn_band3$band <- paste(attr(wd3, "distances"), collapse="-")
sn_band4 <- listw2sn(nb2listw(wd4, style="B", zero.policy=TRUE))
sn_band4$band <- paste(attr(wd4, "distances"), collapse="-")
data_out <- do.call("rbind", list(sn_band1, sn_band2, sn_band3, sn_band4))
class(data_out) <- "data.frame"
table(data_out$band)
data_out$ID_from <- projdata$t50_fid[data_out$from]
data_out$ID_to <- projdata$t50_fid[data_out$to]
data_out$elev_from <- projdata$elevation[data_out$from]
data_out$elev_to <- projdata$elevation[data_out$to]
str(data_out)

The "spatial.neighbour" representation was that used in the S-Plus
SpatialStats module, with "from" and "to" columns, and here drops
no-neighbour cases gracefully. So listw2sn() comes in useful
for creating the output, and from there, just look-up in the
input data.frame. Observations here cannot be their own neighbours.

It would be relevant to know why you need these, are you looking at
variogram clouds?

Hope this clarifies,

Roger

>
> ---------
> Lom
>
> On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]> wrote:
>
>> Roger's example works for me and gives a list of length 101. I did have
>> some issues that were resolved by updating packages. I'm using R 3.6.3 on
>> macOS 10.15.4. I also use rtree successfully on Windows 10 with R 3.6.3.
>>
>> Kent
>>
>> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]> wrote:
>>
>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>
>>>> rtree uses Euclidean distance so the points should be in a coordinate
>>>> system where this makes sense at least as a reasonable approximation.
>>>
>>> I tried the original example:
>>>
>>> remotes::install_github("hunzikp/rtree")
>>> library(spData)
>>> library(sf)
>>> projdata<-st_transform(nz_height, 32759)
>>> library(rtree)
>>> pts <- st_coordinates(projdata)
>>> rt <- RTree(st_coordinates(projdata))
>>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>>> wd1 <- withinDistance(rt, pts, bufferR[1])
>>>
>>> but unfortunately failed (maybe newer Boost headers than yours?):
>>>
>>> Error in UseMethod("withinDistance", rTree) :
>>>    no applicable method for 'withinDistance' applied to an object of
>>> class
>>> "c('list', 'RTree')"
>>>
>>>>
>>>> Kent
>>>>
>>>> On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
>>> wrote:
>>>>
>>>>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>>>>
>>>>>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
>>>>>>> From: Lom Navanyo <[hidden email]>
>>>>>>> To: [hidden email]
>>>>>>> Subject: [R-sig-Geo] How to efficiently generate data of neighboring
>>>>>>>         points within specified radii (distances) for each point in a
>>>>> given
>>>>>>>         points data set.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hello,
>>>>>>> I have data set of about 3400 location points with which I am trying
>>> to
>>>>>>> generate data of each point and their neighbors within defined radii
>>>>> (eg,
>>>>>>> 0.25, 1, and 3 miles).
>>>>>>>
>>>>>>
>>>>>> The rtree package is very fast and memory-efficient for
>>> within-distance
>>>>>> calculations.
>>>>>> https://github.com/hunzikp/rtree
>>>>>
>>>>> Thanks! Does this also apply when the input points are in geographical
>>>>> coordinates?
>>>>>
>>>>> Roger
>>>>>
>>>>>>
>>>>>> Kent Johnson
>>>>>> Cambridge, MA
>>>>>>
>>>>>>       [[alternative HTML version deleted]]
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-sig-Geo mailing list
>>>>>> [hidden email]
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>>>>>
>>>>>
>>>>> --
>>>>> Roger Bivand
>>>>> Department of Economics, Norwegian School of Economics,
>>>>> Helleveien 30, N-5045 Bergen, Norway.
>>>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>>>> https://orcid.org/0000-0003-2392-6140
>>>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>>>
>>>>
>>>
>>> --
>>> Roger Bivand
>>> Department of Economics, Norwegian School of Economics,
>>> Helleveien 30, N-5045 Bergen, Norway.
>>> voice: +47 55 95 93 55; e-mail: [hidden email]
>>> https://orcid.org/0000-0003-2392-6140
>>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>>
>>
>
--
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: [hidden email]
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo
Roger Bivand
Department of Economics
Norwegian School of Economics
Helleveien 30
N-5045 Bergen, Norway

Re: How to efficiently generate data of neighboring points

Tue, 06/02/2020 - 23:20
I had the errors with rtree using R 3.6.3. I have since changed to R 4.0.0
but I got the same error.

And  yes, for Roger's example, I have the objects wd1, ... wd4, all with
length 101. I think my difficulty is my inability to output the list
detailing the point IDs t50_fid.

---------
Lom

On Tue, Jun 2, 2020 at 8:02 PM Kent Johnson <[hidden email]> wrote:

> Roger's example works for me and gives a list of length 101. I did have
> some issues that were resolved by updating packages. I'm using R 3.6.3 on
> macOS 10.15.4. I also use rtree successfully on Windows 10 with R 3.6.3.
>
> Kent
>
> On Tue, Jun 2, 2020 at 12:29 PM Roger Bivand <[hidden email]> wrote:
>
>> On Tue, 2 Jun 2020, Kent Johnson wrote:
>>
>> > rtree uses Euclidean distance so the points should be in a coordinate
>> > system where this makes sense at least as a reasonable approximation.
>>
>> I tried the original example:
>>
>> remotes::install_github("hunzikp/rtree")
>> library(spData)
>> library(sf)
>> projdata<-st_transform(nz_height, 32759)
>> library(rtree)
>> pts <- st_coordinates(projdata)
>> rt <- RTree(st_coordinates(projdata))
>> bufferR <- c(402.336, 1609.34, 3218.69, 4828.03, 6437.38)
>> wd1 <- withinDistance(rt, pts, bufferR[1])
>>
>> but unfortunately failed (maybe newer Boost headers than yours?):
>>
>> Error in UseMethod("withinDistance", rTree) :
>>    no applicable method for 'withinDistance' applied to an object of
>> class
>> "c('list', 'RTree')"
>>
>> >
>> > Kent
>> >
>> > On Tue, Jun 2, 2020 at 9:59 AM Roger Bivand <[hidden email]>
>> wrote:
>> >
>> >> On Tue, 2 Jun 2020, Kent Johnson wrote:
>> >>
>> >>>> Date: Tue, 2 Jun 2020 02:44:17 -0500
>> >>>> From: Lom Navanyo <[hidden email]>
>> >>>> To: [hidden email]
>> >>>> Subject: [R-sig-Geo] How to efficiently generate data of neighboring
>> >>>>         points within specified radii (distances) for each point in a
>> >> given
>> >>>>         points data set.
>> >>>>
>> >>>
>> >>>
>> >>>> Hello,
>> >>>> I have data set of about 3400 location points with which I am trying
>> to
>> >>>> generate data of each point and their neighbors within defined radii
>> >> (eg,
>> >>>> 0.25, 1, and 3 miles).
>> >>>>
>> >>>
>> >>> The rtree package is very fast and memory-efficient for
>> within-distance
>> >>> calculations.
>> >>> https://github.com/hunzikp/rtree
>> >>
>> >> Thanks! Does this also apply when the input points are in geographical
>> >> coordinates?
>> >>
>> >> Roger
>> >>
>> >>>
>> >>> Kent Johnson
>> >>> Cambridge, MA
>> >>>
>> >>>       [[alternative HTML version deleted]]
>> >>>
>> >>> _______________________________________________
>> >>> R-sig-Geo mailing list
>> >>> [hidden email]
>> >>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>> >>>
>> >>
>> >> --
>> >> Roger Bivand
>> >> Department of Economics, Norwegian School of Economics,
>> >> Helleveien 30, N-5045 Bergen, Norway.
>> >> voice: +47 55 95 93 55; e-mail: [hidden email]
>> >> https://orcid.org/0000-0003-2392-6140
>> >> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>> >>
>> >
>>
>> --
>> Roger Bivand
>> Department of Economics, Norwegian School of Economics,
>> Helleveien 30, N-5045 Bergen, Norway.
>> voice: +47 55 95 93 55; e-mail: [hidden email]
>> https://orcid.org/0000-0003-2392-6140
>> https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en
>>
>
        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
[hidden email]
https://stat.ethz.ch/mailman/listinfo/r-sig-geo

Pages