[meta] Create 'Where' field that meets our full complexity requirements
|Assignee:||Sam Boyer||% Done:||
|Category:||dev||Spent time:||2.00 hours|
We want to derive address/geolocative data from a simple text input field, then make it available at view-time either as direct geo info (as in, for use in representing on a map) or as a nice text string describing an address (so, geocoded then reverse-geocoded-normalized). though it might be acceptable to just rely directly on the user input string for the output text...i'm undecided.
Updated by Sam Boyer about 1 year ago
- % Done changed from 0 to 30
ok, 'field_where' (address) has been replaced by 'field_event_where' (text).
there's a LOT more to be done with this. i had some extended discussions with folks in #drupal-geo this morning about this, and i now feel i have a much better handle on the set of issues we need to tackle here. we have two basic options:
- Combine a few existing fields - a textfield input, geocoded storage, and textfield output - to achieve roughly the effect we want, though with some more complex configuration. sounds like we'd have to do a little work on the reverse geocoding front (if we want a nicely-formed textual address as part of our standard output).
- Roll a new field that maybe reuses some of the internal components of geofield, geocode, but ultimately ties it all together into a package with less configuration that achieves this overall result.
i'm on the fence about which to do. the former plays nicer with the community, the latter keeps us a little tighter around the output we actually want. the latter would ALSO probably make it a bit easier to incorporate switching logic that pulls either from the text input box, OR from the venue if one is provided.
either way, here's the full irc backlog for others' perusal:
<sdboyer> the goal is basically to approximate the 'Where' field on gcal - simple text entry field, which we then attempt to geocode, write to another field and then at display time, make structured address + geo info available <sdboyer> on the one hand, it could probably be stitched together from some existing fields, but it seems like the kind of case that might be better to do as a single field that encompasses that whole process, reusing code from geocoder & geofield directly <Brandonian> Out of the box, it's pretty easy to set up a textfield which is geocoded into geofield, which can be setup to display a map either via the built in map or openlayers <sdboyer> or is the marginal benefit there so small as to not be worth it? <sdboyer> map and text display both would be ideal <Brandonian> I wouldn't think it'd be worth it personally, but that's just me, and I'm horriby biased <sdboyer> that's fine, tell me why :) <sdboyer> "explain your bias" <Brandonian> a) geofield and geocoder are relatively lightweight, so having a single field vs 2 IMHO isn't going to gain you much traction <Brandonian> sidebar: You might also be interested in http://drupal.org/project/google_map_field <sdboyer> minimal reliance on google would be preferred, here <Brandonian> b) geocoder/geofield are already coded, get's you 90% of what you want to do (last part would be a "polished" (reverse geocoded) address <sdboyer> yeah ok, i think i see the bias <sdboyer> that is <sdboyer> right now the idea of putting together a field sounds nice to me because this seems like a fairly common case, but one that requires knowledge of how to do the stitching to do <Brandonian> right <sdboyer> however when i'm in situations where i know how the pieces work, the idea of putting something together that's "just stitching" seems quite not worth it to me <sdboyer> but the reverse geocoding may require a little elbow grease? <Brandonian> Honestly, you could probably write what you're wanting to build in a couple of days without geofield/geocoder. You'd want to look at whichever geocoding library you'd want to use. <Brandonian> For example, Google's geocoding library will return address parts in addition to lat/lon, which can be used to populate a "polished" address <Brandonian> And that's a pretty common feature for any geocoding library that I've worked with <sdboyer> wouldn't be a bad experiment from a self-education perspective, i suppose <Brandonian> right, but (selfishly) I'm also looking for people to help us continue to expand our stack, and the more people with knowledge in that stack, the less work i have to do. :-) <sdboyer> :) <sdboyer> and you would consider such a project in the vein of expanding your stack? not forkish-ing it? <Brandonian> If you wrote a field on your own, unless you tied into geocoder or something, it'd be more of a fork. <Brandonian> So, just thinking out loud, if you wrote this on your own, your field(s) would probably look something like... <Brandonian> a - an input address (text field) <sdboyer> ahh i see - so the "honestly..." was the forkish route. the former was describing how i'd play nice with what y'all have done <Brandonian> *read* - yes <Brandonian> b - a way to store lat/lon (either WKT or 2 floats) <Brandonian> c - a renderable address (another text field) <Brandonian> Nicities that you probably would want to look at are caching inputs, or some sort of auto-lookup <Brandonian> (So you're not repinging your geocoder for the same location each time) <sdboyer> yeah that's what i was gonna ask - i imagine we'd cache the results of geocoding & reverse geocoding <sdboyer> right <Brandonian> we sort of do that with geocoder right now, and we're expanding on that to take care of distance searches with views <Brandonian> http://drupal.org/node/1515372 <Druplicon> http://drupal.org/node/1515372 => Caching Results => Geocoder, Code, normal, needs work, 2 comments, 1 IRC mention <sdboyer> eh well, this isn't encouraging: http://www.dotdragnet.com/forum/index.php?PHPSESSID=3078f202d4021ecf923fecd229291089&topic=3983.msg30721#msg30721 <sdboyer> ah i see it's linked to in that verypost <Brandonian> Thought of that. Terms of service basically say if it's for performance reason's, you're good <Brandonian> Which is what we argue <sdboyer> yep <sdboyer> makes sense in the context of that issue, where you're performing a search <sdboyer> would be more questionable for the reverse geocoding, i think <Brandonian> Probably, but if nothing else, it's only 1 ping instead of 2 <sdboyer> ps, i agree - create a separate cache bin for geocoded data <sdboyer> true <sdboyer> seems bizarre to me that we don't just cache that data arbitrarily <Brandonian> And if we already have the data linked, then we can just pull from the local <sdboyer> other geocoding providers don't have such a limitation, i take it? --> phayes (~anonymous@hw-phayes-eth.Stanford.EDU) has joined #drupal-geo -*- sdboyer waves at phayes <phayes> Oi! <Brandonian> not 100% sure, but google/yahoo probably both don't want someone to set up a geocoding service using their data <phayes> Correct <Brandonian> which is what I think they're trying to get at <sdboyer> ahhh, that makes sense <sdboyer> yeah i can see how interminable caching of results amounts to just sucking all the data off their service <phayes> context to discussion? <Brandonian> phayes: I'm trying to talk sdboyer into using geofield/geocoder instead of rolling his own field for a project <sdboyer> :) <Brandonian> and by extension, contributing to the greater good <Brandonian> :-) <phayes> Yup yup. very good ideas <sdboyer> contrary to my reputation in certain drupally circumstances, i actually DO like playing well with others when it comes to code :P <phayes> Geocoder actually has a REST / AJax API ready to go <sdboyer> phayes: http://pastebin.com/6M1T0QUA there's more context <Brandonian> It's kind of a sore spot in the geospatial/drupal sub-community. We all like to roll our own at one point or another <sdboyer> it does seem tricky <phayes> sdboyer, does it need to be interactive? <Brandonian> It can get very complicated very quickly <sdboyer> there are a lot of different use cases for fields, widgets & workflows, and managing all those thinks sanely together ends up being a combinatorial complexity mess <sdboyer> phayes: define "interactive"? as in, on-demand geocoding of the textfield you're typing, as you type it? <sdboyer> s/on-demand/AJAXy/ <phayes> sdboyer - correct <sdboyer> i'd call that a nice-to-have, not a requirement --> mokaspar (~Adium@adsl-84-227-28-18.adslplus.ch) has joined #drupal-geo <phayes> ok cool <sdboyer> we have an interesting problem with the results caching, though --> kricklin (~firstname.lastname@example.org) has joined #drupal-geo <sdboyer> the long-term plan for how this data will be reported has the geocoded results being placed into a triple store for potentially long-term caching <sdboyer> or maybe just more importantly that the geocoded data will be emitted via RDF, so it is reasonable to expect that it will be picked up by such stores <phayes> sdboyer - geocoded provides a link to handler's "use policy" so that site builders can judge for themselves wether they are able to use the handler in a compliant fasion <phayes> n <phayes> We just provide the functionality, the link to the policy, and let the admin decide <phayes> For yourself, I suspect you would be in violation of google's policyu <phayes> I think yahoo is more lenient <sdboyer> phayes: that's good, but isn't sufficient for our case, since we're looking at building a more complex distro + platform <sdboyer> yeah <sdboyer> if we can get away with it just by restricting providers, that may be sufficient <phayes> I would also look into building a bing handler for geocoder <sdboyer> do y'all use ctools plugins for the provider handlers? :P <phayes> we do! <sdboyer> woot :) <sdboyer> OO or functional? <phayes> functional <sdboyer> i'm sorry in either case, just for different reasons <phayes> just listing callback functions in the plugin array <sdboyer> yep -*- sdboyer is pondering if google/any provider could reasonably bring suit against a downstream RDF triple store... --> steinmb (~email@example.com) has joined #drupal-geo <phayes> I doubt it would *ever* come to a suit <phayes> google tends to be pretty polite about license violations <sdboyer> of course, threat of suit is what matters <phayes> but might get a slap not he wrist <phayes> fight <phayes> right <sdboyer> :P <sdboyer> we cache their data, and set some reasonable TTLs (say 86400s) on the page/rdf resource, which includes their geocoded results <sdboyer> and the downstream provider picks it up...but say they arbitrarily decide to increase those TTLs to 7d from 1d <phayes> I would honestly research yahoo, bing, and MapQuest to see what their terms are <sdboyer> who gets slapped? <sdboyer> yeah <sdboyer> we'll have to <phayes> MapQuest uses openstreetmap data <phayes> so that one is the most likely to have good terms <sdboyer> noted <zzolo> phayes: and sdboyer, map quest as a very open geocoded. the issue with any non-google geocoded is that its just not that good. bing's is probably second best <sdboyer> zzolo: yeah, quality was the next place i was gonna go <sdboyer> zzolo: ps, HIHI <zzolo> hey :) <sdboyer> how MUCH worse...in wonderfully subjective terms? <zzolo> overall, i would suggest Bing for the next best accurate solution, but MapQuest for the best open solution <sdboyer> yeah we're prolly gonna be bound to the best open solution <zzolo> sdboyer: its hard to say. for instance, with MapQuest, if I put in just "San Diego" its not going to assume a state and i won't get back a result. But, "San Diego, CA" will work --> plopesc (~firstname.lastname@example.org) has joined #drupal-geo <zzolo> sdboyer: like with Google, you can put in " <zzolo> "SF" and it knows its san francisco <sdboyer> can't afford to put something out there that gets occupations in hot water without them knowing why <sdboyer> zzolo: yeah, google is just stupid smart <zzolo> its hard to say if it actually tries to look at where you are to figure that out <sdboyer> i've always felt like it has <zzolo> or if it just has "hardcoded" values and machine learns what people are looking at
Updated by Sam Boyer about 1 year ago
- Subject changed from Replace address-type field with simple text field in 'Where' to [meta] Create 'Where' field that meets our full complexity requirements
i'm hijacking this issue to make it into a meta-issue for the overall set of requirements we have for the 'Where' field.
Updated by Andrew Mallis about 1 year ago
The reverse geo-coding piece is the hard part, and gets increasingly more complex when you think about addresses being outside the US.
Geofield's got some good geocoding happening for sure.
Here is a possible sketch of the Venue geocoding behavior, when adding a Venue to an Event:
I type an address into a single text field and press the "look up" button.
The address is geocoded via ajax and returns a result
The result is checked against the database of existing locations
The address is parsed into individual, editable and labelled address fields below the map
The look up button is relocated by the address fields and will geocode again based on values entered there if pressed
A map is displayed with a marker (user can move the marker if desired)
Other Venues are also displayed on the map (with subtle markers) and are listed beside the map.
Venues which most closely match the new entry (geospatially, and by category) float to the top of this list and are highlighted suggestions
If there is a positive match, a message is displayed to the user more explicitly suggesting the existing Venue.
Pressing the "use" button accompanying each suggested Venue will swap out the address field values, and the marker type
The Submit button saves the location in all its glory
There are some holes here, still. For example, maybe there is an Event already using a Venue, and they have an apt # listed there. Do we want that blown out here or replaced with the new event's apt #? Probably not.
There's no way we'll be spot on reverse geocoding from a textfield, so we'll want to allow for some adjustment before submission, and also want to assist the user.