RESO 2013 Fall Meeting Liveblog

This is a live blog and will be updated over the course of the 2-day meeting. Some of this won’t read easily if you haven’t been involved in RETS for a while.

Business Update

Bob GottesmanThe next RESO meeting will be in Chicago (at NAR) April 23-25, 2014.

If you want to be on the Board of Directors, look for the nomination form on the RESO website. Nominations are open until November 16. Online elections December 9-20. Terms start January 1.

Membership funds the efforts. Currently membership is as follows: Class A – NAR, Class B – 37 Vendors / Consultants, Class C – 52 MLSs, Class D – 3 Brokerages.

Legal Update

The Intellectual Property Rights Policy is ongoing – mostly working on the definition of what RESO will do and to limit the liability of RESO while being fair for members. The overall goal is to make sure that members have rights and permission from their organization or online sources for anything they contribute to RESO, to make sure the IP is flowing properly. RESO members are currently working under the existing contributor license.

The Antitrust Policy is there to keep us all safe: Don’t talk about prices or fixing pricing, or share information on pricing or costs. Don’t agree with another participant to boycott a group or treat a group of suppliers in a specific manner. Don’t agree to allocate customers or territories. Don’t complain about vendors or a class of vendors or their outputs. The board will soon provide a more comprehensive anti-trust blurb to provide to workgroup participants.

RESO has the words RESO and RETS trademarked. They will be filing for federal registration of word and design marks, will defend those marks and that will tie into compliance. “RETS” is trickier as it’s more descriptive and restrictions on use will be status quo.

State of the Standards

RESO now has several staff people (some part time) to assist Bob G. This should help RESO operate better ongoing.

RETS will be the existing 1.x protocol being used today. New products/services – like the new ‘transport’ will not be called RETS. Matt says: This has implications for MLS contracts – those might have to be amended to ensure ongoing compliance with “any standards created by RESO”. The funny thing is one MLS Exec said: “My vendor wouldn’t charge me more if the name was changed,” and another vendor immediately said, “We would totally want to charge more.” So, this is serious. This was discussed at length to see if perhaps “RETS” could be amended in some way to make it trademark-able. TBD. It’s also important to keep RETS as the umbrella name because everyone knows the name RETS. Again, this is going to be discussed ongoing. There’s time to ponder it.

The “data dictionary” is so important – this needs to be communicated to the industry.

The charter and vision: RESO will develop, adopt, & implement open standards that

-          facilitate software innovation

-          eliminate redundancies

-          ensure portability

-          create maximum efficiencies

RESO will be sensitive to industry needs on a timely basis (balancing speed with a deliberate standards process). RESO will promote standards and membership (participation is as important as the money). RESO will improve infrastructure (tools to make the work groups more productive). RESO will make itself relevant.

Major actions include:

-          Data Dictionary (6 MLSs have mapped to it. Now it’s time to migrate it to production.)

-          Certification and Compliance (adding meaning and credibility)

-          RETS 1x vs. Transport (is there really a battle? [No - the new transport is for 'lightweight' uses])

-          Improving workgroup productivity (new collaboration system)

One of the most important issues is to get completed actions (standards) to be put into practice. Early adopters will be recognized and awarded.

Mark Lesswing: RESO’s Data Dictionary, API and Event Catalog

Mark LesswingMark says we need to add an “Event Catalog” to the mix (of Data Dictionary and API Transport) – to help create analytics. An event catalog documents actions performed by users, information to “fine tune” an action, and data that results from an action. Right now we’re going to talk about this in terms of analytics for MLSs (not for others). This will let MLSs anticipate the needs of their customers.

Events could model a lifecycle – including: CreateListing, RequestShowing, UpdateListing, TakeOffMarket, MakeOffer, Closed.  We could create predictive analytics from this.  Imagine resource consumption inside the MLS being chartable – PropertyDetail, GetHotSheet, PropertySummary, SearchUsingLocation… it could show where a service is being used (geographically), what is the next event in the lifecycle? Where are listings being taken? Could this be useful in determining where service centers should be?

Ed Newman suggests we need an API / Event model to get tax data from counties – instead of bulk tax data. Matt says: awesome idea. Good luck!

Ira Luntz: “Why RESO?”

“RESO sells houses”.  Well, it helps sell houses.

Everything is easier when the same standards and data dictionary are used to develop products and services, because everything works together and we’re all talking the same language.

No one else in the industry has the depth and breadth of RESO – RETS 1.8 and the Data Dictionary are the industry’s best option to define and transport quality data for the future.

RETS and the Data Dictionary is the foundation and wellspring for the most innovating products, apps and services that will help the real estate industry thrive, regardless of market conditions.

RESO is a vibrant and powerful community of expert developers and business partners to spur the adoption of RETS 1.8.

When the community works together, better products are built, integrated and delivered to our ultimate customers: real estate professionals and consumers.

Certification & Compliance Systems

A certification system will verify systems claiming to be RESO/RETS compliant

Project goals: Add credibility and value to RESO/RETS standard. Recognize RESO/RETS compliant, Make RESO nationally relevant.

Previous certification efforts started in 2008, Hitachi re-wrote it in 2009.  There are issues, and the project has been on hold. Ronin Technologies is going to help bring it back to life.

Who applies for a certificate? RETS Clients, RETS Servers, Sites & MLS. A site is someone that is responsible for the construct of the data and owning that data. Right now 99% are MLS but it could be an entity that’s not an MLS. It’s the MLS that applies – NOT their vendor.

Awards: Pass or No Pass (no reported failure). Certificates, Badges, Logos & Mentions (recognition beyond the certificate test).

Compliance testing is planned for RETS 1.8 Client & RETS 1.8 Server, Data Dictionary for RETS Version 1.5-1.8. Future certificates will be for OData / REST API.

Certification will cost money, but the goal is break even. Members will certify at a discount.

Data dictionary certification will test site/MLS metadata against the data dictionary. Non-data dictionary fields in the site/MLS metadata are ignored. There will be certification on multiple RETS standards – 1.5-1.7.2, and 1.8. The exact implementation is TBD. There will be one data dictionary certificate with additional special mentions are still being discussed. The process, at a high level, is as follows: There will be pre-test tools that can be downloaded for testing. There will be both automated and manual testing. The certification team will make a compliance decision based on test results. The applicant can fix the issue or appeal if there is a negative result. If there’s a positive result, the applicant will receive a certificate and information will be posted on reso.org.

The goal is to have Site/MLS & Client certification ready in January of 2014. The data dictionary certification date is still TBD.

The cost for RESO members will be “three digits” – for non-members, more. It’s still TBD, depending on RESO’s own costs. How often re-certification happens is also TBD.

There’s some discussion of RETS Update certification – Bob says, “If there’s demand it could be done in 2014″.

Paula O’Brien: Compliance testing has been integral since 2003. It helped drive 1.5 adoption. There’s an immediate need for 1.8 testing to drive adoption. Tests for the dictionary as our ‘lingua franca’ is needed. The challenge is to be agile and quick to implement but have thorough tests. Paula’s goal is to resurrect the existing working frameworks and update the test scripts and reports. It was developed originally in Java as open source and will remain so, so it can be downloaded and used by end-users. The big key is to keep it flexible so new tests can be dropped in.

There has been a call to bring back ‘plug fests’ – where we can sit around and make sure products work with each other in real life.

Paul Desormeaux: R & D – 2014 Potential Projects

Paul DesormeauxR&D filters ideas from members and through the board facilitates workgroup formation. Member input is critical and appreciated and members are encouraged to join the workgroup.

A survey was done by the R&D group to suggest workgroup collaboration systems. 22 products were reviewed by a group of 3. Collaborative documentation was a critical feature for the selection – also, file sharing, versioning, notifications, full-text search, cross-browser, assigning individual and group privileges. Hosted Confluence was selected as a collaboration system. The group is looking at add-ins – task lists, schedulers. Hopefully we can put everything in one place (versus the way it is now).

Looking at the business case master list, following are very active:

  • Standardizing (listing source) organization names
    • Important for reducing duplicates in listing statistics and unique listing queries (inter-operation API and more). RESO would have to maintain a unique organization ID, full organization, and a short name (duplicates allowed). The issues: who keeps it updated, what about the persistence of entries (what happens when there’s regionalization), update process and frequency, and what is the delivery mechanism?
  • Property unique identifier (PUI)
    • Important for reducing duplicates in listing statistics and unique listing queries (inter-operation API and more). Many formulas are being considered. Country Code + State/Province + County + Parcel # / PIN? Or replacing Parcel with geolocation?
  • Data Replication
    • The current focus of RESO standards – light-weight efficient JIT fetching of data (mobile) and discouraging full copies of listing DBs (though there’s still a strong need for replication). Issues are data level vs transport level (not always done right at data level with timestamps).  New OData V4 includes change sets  – something that will be looked at.
  • Looking ahead
    • Business Rule Payload
      • The idea is to expose usually hidden business rules to answer “Why did I not get all the records back?” It’s tricky to do with the current RETS Validation BNF.
    • Events catalog
      • As per Mark Lesswing’s discussion.
  • Updating the RESO Forum at reso.org

Rolling out Data Dictionary

The migration work has been working with six volunteers to roll out the data dictionary: ARMLS, CRMLS, Metrolist (Sacramento), MLS Listings, Inc., MRED, and TREND. Their mapping is complete – next they need to analyze and plan for production and get clients to hit it. The challenges are mapping local metadata to the data dictionary, analysis and planning for production, implementation, and encouraging adoption. [The group then reviewed a few typical mapping issues].

RESO Workgroup Community

Presented by: Marcy Foster (ARMLS)

There are problems: disparate work areas (reso.org, Google groups for workgroup conversations,  Google Drive for file creation, maintenance and storage, and Confluence for standard document drafting, commenting and storage). The goal is to create ONE consistent, easy to remember environment – reso.org.

The requirements: document management, site publishing, project management, collaboration tools, security, device accessibility, product support, and price.

There was a survey – 36% responded. 22 products were identified. 6 made the short list, but Confluence was picked, along with some plugins that extend its capability. RESO is currently only using 5% of its capabilities. No other product came close to comparing on the primary need: document management.

Each workgroup environment will have a home page, workspace pages, a file manager, discussion board, task list, and meeting notes.

All RESO members will have access to ALL Workgroup areas – this will enhance cooperation and reduce discussion “silos”.

Paul Stusiak: RETS 1x Workgroup

The workgroup is there to receive issues and enhancement requests, to prioritize issues and enhancements, and to discuss and recommend solutions to issues and enhancements.
Currently the group is working on using the data dictionary in RETS 1 (See Developer Note 003 on reso.org – there are two proposals for how to do this in the note), the “killed” resource (to know what’s not available to a RETS user any more and why – compared with grabbing a whole new set for replication and comparing that against what you already have in your database), and forward enhancements to the dictionary – how to deal with standard lookups  / enumerations.

Scott Petronis & Matt McGuire: Transport Review and Status

scott petronisThe objective is to recommend an approach for a RESTful way to deliver real estate data, to enable direct interaction with data from web, mobile, and social applications.

Design criteria are to keep things lightweight and not to re-invent the wheel – to use existing open Internet standards. This boils down to Dictionary, Transport and Security: these three areas together create the platform, which can be certified.

The process is to research, review options, debate alternatives, recommend a path, document, have RESO review / comment, ratify, and move forward!

The group has collected ~60 use cases, compared to OData, found an established path with OData v3, and has started drafting documentation. This should be completed over the next month.

The scope will be search focus (http get): metadata representation, read access / standard search, limited geospatial search, and hypermedia representation. Out of scope in this initial release will be create, update and delete functionality, a data replication framework, updating binary media resources, and saved searches and resources. A lot of these out of scope items are certainly possible within the OData standard. In terms of resources (payloads), the initial focus is on listings, members, offices, and media. Others can be added.

In terms of specific query strings, $select, $filter, $top MUST be supported. $skip and $orderby MAY be supported. In terms of operators, there is support for logical (and/or/not), equality (eq, lt, gt, ge, …), string (substring, startswith, endswith…) and enumeration. There are geospatial functions following OGC specifications. It supports key primitive data types – points, polygons, multipolygons and more. Core functions include geo.distance and geo.intersects.

The next steps are to finalize documentation by the end of the month, put documentation out for review in November, revisions in December, ratify possibly by end of year(?), and prototype in Q1.

Michael Wurzer: Interoperability

This workgroup’s goal is to recommend ways to pass data between applications using a standard method. Some examples:

Listing IDs – i.e. myapp.com/?LaunchListingIds=”ListingId[ListingId(0-n) [, ListingSource]

Listing search – i.e. myapp.com/?LaunchListingSearch=TotalBedrooms%20Gt%204

Saved search IDs – i.e. myapp.com/?LaunchSavedSearchIds=200060412165917817933

[a bunch more]

There will also be a way of passing application metadata to determine what kinds of app something is and what inputs it can accept.

The workgroup’s documentation will go to the board, which will publish it for review.

This may be published as a “best practice” or a “standard” (TBD).

Rob Larson: Data Dictionary

Dictionary 1.2 has been published, including open house, saved search, 67 new enumerations and 78 changes (mostly additions and a fix to repeating elements). The dictionary (1.2) now covers 6 property types, member office, contacts, saved search, open house, history (transactional) and 132 enumerations.

Version 1.3 is coming this spring, covering commercial (resimercial), more enumerations, and (via technical subcommittee) queue resource (track and communicate replication success – transport independent), kill resource (as previously discussed), [prospects, activity, events], and spreadsheet to schema.

In 2014: working on business rules, response to client upload (1.x workgroup), communicating rules to client in advance.

Matt Cohen: Authentication & Authorization

matt cohen at resoMission: “this group was formed to create “a cohesive [authentication] strategy and facilitate the use of recognized protocols. This group is “tasked with researching existing industry methods and recommending the adoption of appropriate and proven technologies.”

Immediate Goal: To define authentication methods for RETS, primarily for a new (RESTful) API
The goal is NOT:

  • To solve the problem of managing passwords
  • To decide implementation specifics not germane to secure interoperation
  • To provide mechanisms for Single Sign-On
  • To provide mechanisms for Federation of Identity

However, standards needed for our goal of secure RETS “back end” authentication may also have “front end” capabilities such as supporting SSO.

Four use cases:

1. SP to SP/IdP* – Server-to-server authorization (see Google’s “OAuth for Service Accounts” or more canonical writings on “2-legged OAuth”).
Example:  A syndicator’s recurring bulk download of listing data.

2. SP to IdP to SP – Typical three-way authorization of a user.
Example:  A web application that interacts with the MLS, e.g., a real-time CMA.

3. SP to SP/IdP – Transparent three-way authorization of a user.
Example:  A VOW provider’s validation of eligibility for an existing customer.

4. SP to SP/IdP – Transparent, recurring “on behalf of” authorization of a user.
Example:  Lead Management software that pulls leads from multiple sources for a given customer.

Standards being considered:

  • Digest
  • oAuth 1
  • oAuth 2
  • oAuth2 with SAML 2.0 Bearer Assertion
  • SAML
  • OpenID Connect

Not every approach will work for every use case.

In progress – Document:

  • How does standard cover use cases?
  • Provide Technical Examples (show calls)
  • Suggest “options” within standards
  • What toolkits are available?
  • Pros and cons?

Cal (FBS) has started an “advocate” document for oAuth2.
Others have been solicited as per survey results

TO-DO – Select one or more approach. Evaluation Criteria

1.    Addresses Use Cases
2.    Ease of use
3.    Use of existing standards
4.    Use of (real estate industry) deployed standards?

TO-DO – Interoperability requires more “musts” than “mays”. Reduce choices for implementing the more complex standards.

James Martin: Syndication

Usage of the syndication standard is stable – over 130 companies receiving it. Users include national publishers including most from the top 10, regional/niche publishers, and franchises. The standard hasn’t changed in a while but that’s going to change, as syndication receivers want more data.

The group is currently looking into data dictionary alignment. Looking at 387 fields, 11% need to be renamed, 5% added, 3% keep as is, 24% add to the dictionary (!), 22% drop, 15% NoOp, and 20% TBD.

Fall 2013 enhancements include lead routing additions, lead status alignment with the data dictionary, and MLS logo addition.

Bob G.: Day 2 Introduction

Some of the tools – like LibRETS are going to be updated soon.

RESO will start writing “best practice” documents in addition to “standards” documents. For example, the interoperability ideas described yesterday may start out as a best practice document and – if it does well – it could be turned into a standards document.

Planning and Strategy for Workgroups

We want to generate more workgroup participation. It’s a problem that many people are in many workgroups – and many in no workgroups. Folks on the sidelines – please come in!

We want to share ideas. R&D is a great place to do that.

Authentication: Please get your authentication standard “advocacy” documents in. We’ve already had folks step up and volunteer to write some additional ones (Digest, SAML profile for oAuth).

Dictionary: There may not be a problem regarding booleans. If we are going to have booleans, they need to be nullable. Otherwise they need to be a single select string. Preferences? Boolean / nullable seems to be preferred and it will be implemented in version 1.3, up for comment next year. Can we change the “kill” resource to “hidden” or something else? No one is opposed – TBD.

R&D:  There are two items to tackle this fall: The first is Organization ID – we’ll be in touch w/ RPR but we’d like people to help in a sub-workgroup for this. How much information should we track? How long to keep it? Is there a document or RESO web service?  The second one is property (not listing) unique identifier. What we want to create is an alogorithm versus a lookup. In 2014, there are two more. The first is business rules payload – to find out why you’re not getting listings on a query due to business rules. Also, so MLS staff can look at their business rules. The second is Mark Lesswing’s “events catalog”. People need to get involved with that.

Transport: We’ve got work cut out for the next few months. At the end of this month the group will get together in person and get documentation done for comment. Then we will prototype – we want to know what people want from that. Server side? Client thing? Interaction with HTML 5? Expecting that will happen in Q1 (a tall order, but we’ll figure that out). A lot will be left for the second phase, and we’ll make a roadmap so people know what’s coming  – otherwise there will be more divergence. Participate!

Last Words

There was an EXTENSIVE conversation regarding whether it’s worth it to put a lot of effort into “Update” and “Business Rules” – will brokers really step up to use it? Will they be willing to build the kind of business system that would create the accurate listings needed to get through validation and put listings into the MLS? Vendors are trying to figure out whether its going to be worth the effort currently planned by many to implement update – with an improved business rules engine RESO is working on.

Should brokers start sending their technical staff to RESO? And/or do MLS RESO participants need to go back home and talk (next week) to their brokers’ CIOs and vendors.

Please visit www.reso.org for more information about RESO and RETS. If you’re not already a member – PLEASE join and get involved in the standards effort.

TTFN!

- Matt

Matt Selfie

Tagged with:
Posted in Matt's Real Estate Technology Blog
4 comments on “RESO 2013 Fall Meeting Liveblog
  1. Travis In TeXas says:

    Matt, thanks for the great reporting. Feels like I’m there!

  2. Ditto Travis. Matt did an excellent job!

  3. Was about to start reviewing, cleaning-up and summarizing my RESO notes … Ugg… But see that that Matt has already done a great job of this. Not sure how he all this so quickly & effectively, but thanks. Just the right level of detail.

    PS : Reminds me of something i have thought about for years :: Some sort of automatic / semi-automatic means of bringing together notes from multiple participants, summarizing them into an integrated, rich, more complete & detailed whole. Don’t think that exists – Maybe I need to start coding … :-)