E&0 - Editorial and Opinion
This Month’s E&O is written by Cal Carter, a REALTOR selling homes in Alabama’s Gulf Coast.
Have an opposing view? We welcome it! Please contact us!We want your voice to be heard.
Have you examined the irony of the MIBOR and NAR policies on IDX Feeds in regard to indexing by Google?
Sometimes I want to relate to my peers some of my thoughts and I choose words accordingly to better relate those thoughts. As I was thinking on what to include in this post, the word “Irony” popped into mind. We all use words that we know what is meant by them, but couldn’t necessarily provide a precise definition. I often go to wikipedia to make sure the language I want to use properly relates to my thoughts so that I don’t end up looking like a moron to the reader. So, as I sit down to write this I will put before you the wikipedia definition of the word “Irony” -
Irony (from the Ancient Greek εἰρωνεία eirōneía, meaning hypocrisy, deception, or feigned ignorance) is a literary or rhetorical device, in which there is an incongruity or discordance between what one says or does and what one means or what is generally understood. Irony is a mode of expression that calls attention to the character’s knowledge and that of the audience.
There is some argument about what qualifies as ironic, but all senses of irony revolve around the perceived notion of an incongruity between what is said and what is meant, or between an understanding or expectation of a reality and what actually happens, “when the literal truth is in direct discordance to the perceived truth.” - http://en.wikipedia.org/wiki/Irony
Is Google an Index of unique content or is it a scraper?
If you have been keeping up with the saga of Paula Henry and her brokerage Red Door Real Estate in Indianapolis, Indiana, you would know that the MIBOR (Metropolitan Indianapolis Board of Realtors) has issued a cease and desist order preventing Paula and Red Door Real Estate’s websites from allowing indexing by search engines. You may read all about this issue and the uproar that it has created here on Paula Henry’s post titled “Did Google Scrape My Website, You Be the Judge“.
As you will observe from the more than 380 comments that the post generated is that the definition of “scraping” was not properly defined in the original policy and in fact, the word “scrapping” is used on the policy. For all of you guys that are familiar with sites being “scraped, by scrapers” you all know that your original content is being taken and used by others with no agreement nor permission to do so. As far as being “scrapped, by scrappers”, I don’t really know what that means, however the policy states that we must “protect the IDX from un-authorized scrapping”. I assure you, I will do my best to prevent my site from being “scraped” or “scrapped” by unauthorized entities. Just as I have done in the past with “copy and pasters” that have stolen content, I will send a polite note requesting that the offending content be removed and escalate demands if necessary.
But is Google a Scraper or Scrapper?
Google is an “Index of Content on the Internet”. Just like all computers on the internet, the bot or spider reads a copy of the code that “is the website”. It places this code in a “cache” on the computer that is reading the web page. It is not “scraped” or “scrapped” content, it is “the content” of the web page. If you will look on your own computer cache, you will find that you have a copy of this web page on your computer in a cache. Are you a “scraper” or “scrapper”?
Ironically I received an e-mail from an attorney that is in my lead management system today. The following was on the footer of hos communication:
SECURITY NOTICE TO CLIENTS - If you are a client of this firm, please be advised that (1) E-Mail communication is not a secure method of communication; (2) any E-Mail that is sent to you or by you may be copied and held by any or all computers through which it passes as it is transmitted; and, (3) persons not participating in our communication may intercept our communications by improperly accessing either of our computers or another computer unconnected to either of us through which the E-Mail is passed.
For those of you that have not figured it out, the same holds true for all the webpages that are visited. They are held in a “cache” on computers in many places. I have written about the issue here - MIBOR and NAR have no grasp of internet technology and how it works.
Block the Spiders with the Robots.txt file!
It has been suggested that the simple fix is to block “scrapers” and “scrappers” with the robots.txt file. The irony here is that yes, it will block well behaved spiders from indexing sites. Search Engines like Google have well behaved spiders that will read the robots.txt file and obediently keep away from the contents that are “noindexed”. However, the true “scrapers” and “scrappers” simply ignore the robots.txt and “scrape” and “scrap” at will. Just as the attorney who’s footer warns against using e-mail if you are worried about the content being read by others, one should never ever place content on the web if they are concerned about the content being “crawled, indexed, scraped, or scrapped”.
IFrames are the solution!
That is a joke. IFrames are a means to display a website within a website. Search engines typically do not read within an IFrame. However, just one link to the website that is within the frame and it too will be “crawled, indexed, scraped, and scrapped”. Again, if you don’t want something to be found on the internet, don’t put it on there! Review this post again, I have documented that IFramed IDX solutions are crawled and indexed.
So what is the solution to this IDX conundrum?
NAR (National Association of REALTOR’s) flew Paula Henry (the victim of this witch hunt) and Jay Thompson (a well respected REALTOR and internet authority) to Washington to testify before the NAR National MLS Committee’s mid year meeting. You see, MIBOR based their “cease and desist” order on an NAR opinion that “Google is a Scraper / Scrapper”. After this firestorm of controversy erupted in the comment section of Paula’s original post, NAR decided to fly them in and smooth some feathers of a very upset and substantial member base of influential tech savvy REALTOR’s.
Actually, the hearings went better than expected and the MLS Committee essentially came out with new language that stated:
However, when the new language was sent to the NAR Board for a vote, the language was tabled at the request of none other than a MIBOR member. Even after Paula Henry was told by a MIBOR member that there was a “willingness to incorporate the rule as soon as it is approved”. Did you get that Paula - “as soon as it is approved”! No wonder you were shocked when the policy language change was tabled - perhaps the MIBOR back room policy is “when pigs fly!”
So here we are with huge investments of time and money by proactive individual agents that are having those investments confiscated for the benefit of an individual complainer!
Enter the NAR Social Media Mogul to resmooth the feathers of progressive agents
As this story breaks out of the Real Estate community and in to the search engine community some more interesting posts pop up. In fact, the post writers are quite knowledgeable about the internet, content ownership, indexing, scrapeing, cacheing, and so on - yet they know nothing of the IDX agreement that all agents and brokers must comply with. The only negative points in the articles they write is already covered by IDX agreements and is a non-issue. http://www.bruceclay.com/blog/archives/2009/05/google_threaten.html
Unfortunately in my opinion is that Todd Carpenter the NAR Social Media Czar must have had a “Google Alert” go off when the Bruce Clay article was written and felt compelled to come to the defense of what was happening. Todd is a great guy and is doing his job, but his response in the comments was:
I’m the social media manager for the National Association of Realtors. As we have continually remarked in other blog posts covering this story, we are not classifying Google as a scraper. The way in which the IDX provider is publishing content (so that Google can read it) opens the door to scrapers as well. That’s not in compliance with the IDX policy voted upon by our members.
Todd’s short statement says that you “can”, but you “can’t” and NAR says so. Rest assured that this must be exactly what one of the MIBOR members and one of the NAR Committee members wanted to hear.
The flaw here is that individuals are being singled out as the “Big Bad Wolf” for doing a good job in their efforts.
A reader of the commentary in all of the posts will see over and over that indexing of the websites and IDX content is to the benefit of:
- Buyers
- Sellers
- Buyers Agents
- Selling Agents
So who is complaining? Perhaps it is agents that want both sides of a transaction to the detriment of seller and buyer representation. Yes, you know, either an unrepresented buyer or the tricky “dual agency”. It smacks of a return to the day of everyone representing the seller. In my opinion is not good and it is not healthy. After reading all of these posts and the hundreds of comments that accompany them though, one can only come up with the fact that anyone opposed to IDX listings being indexed by Google is practicing self interest above the interests of “Sellers”. If this was not the case, the complainer would be elated that his listing was showing up in multiple places on the search engines.
You can’t advertise another agents listing without permission!
Well, for anyone that was licensed to sell real estate this is a known quantity. But is an entry in a Google “snippet” an advertisement. No! It is nothing more than a small amount of text that lets the searcher know if the search result is relevant to what they were looking for. There is not enough information there for it to be an advertisement. None the less, if you want to argue that it is in fact an advertisement, then be cognizant that both the listing agent and listing broker have approved doing so by inclusion of the listing in the IDX. I think my friend Bob Crain from San Diego did a fine job of explaining this in his response to the Bruce Clay post:
Like Cal, I would like to clear up a few misconceptions that are out there… First, concerning the MLS. Most Consumers AND Agents don’t understand what the MLS is.
“The Multiple Listing Service is a means by which authorized MLS broker participants establish legal relationships with other participants by making a blanket unilateral contractual offer of compensation and cooperation to other broker participants; by which information is accumulated and disseminated to enable authorized participants to prepare appraisals and other valuations of real property for bonafide clients and customers; by which participants engaging in real estate appraisal contribute to common databases; and is a facility for the orderly correlation and dissemination of listing information among the participants so that they may better serve their clients, customers and the public.” - http://www.sandicor.com/rules-regulations/2007-APPROVED-RULES0808a.pdf
Therefore, the MLS was established as a means for agents to cooperate with other agents. It is an agent to agent offer of compensation and Co-Operation.
What it has become with the growing popularity of the internet… It has become a means by which internet savvy Brokers and Agents can set up virtual offices to compete with the traditional Brick and Mortar Real Estate Brokerages. This is a threat to those Brick and Mortar Businesses
What it is NOT… It is not, and never was intended to be, an advertising tool for listing agents to get more buyers to call them directly (and potentially NOT be represented by a Buyers Agents). Remember it was and still is simply an offer for compensation and Co-operation between agents.
As Cal stated; the consumer does NOT benefit when they are NOT being properly represented (in my opinion Dual Agency is not in anyone’s best interest except the Listing Agent).
What is Dual Agency? Dual Agency is the listing agent attempting to represent both the buyer and the seller. Folks, the listing agent has a responsiblity to get the most money they can for the Seller. The only incentive they have in working both sides of a transaction is to Keep ALL the Commission (P.S. in almost every case there is no reduction in the amount of commission if the buyer also uses the same agent as the seller).
To all you buyers out there - the total commission the seller pays has already been determined by the listing contract. The buyer thinking that they may save some commission is one of the buyers biggest misconceptions.
Finally, to Todd
Todd – do we really need another lawsuit from the Department of Justice in order for NAR to do the right thing here? http://www.usdoj.gov/atr/public/press_releases/2005/211008.htm
Heck, if NAR really wants to end these silly squables then the National Association of Realtors should outlaw Dual Agency.
In many of the comments you will find sane agents that state “If you want to advertise my property, go ahead - I am trying to sell my listing clients property!”
Now for the real Irony!
This morning I received a nice e-mail from an agent in a neighboring community that told me how nice my site was and that she would be visiting it again (yes, I have agents that use my site to search for their clients). I wanted to know more about her and Googled her name and city. No result. I Googled her brokerage name. There was no site for it, but I did get a hit for her name and brokerage on this IDX site - http://www.weichert.com/20949449/?ldview=ldsummary . It struck me that here was a nice agent, working for her clients that really had no search engine exposure! Did she or her broker want it? I am sure so, but the quality and relevance just must not be there for Google to consider them in their search results. Perhaps they just aren’t tech savvy enough! However, their client is in good hands for her listing is showing up on an IDX site and coming up in the search engines. A win - win!
Upon further examination I noticed that the site was a national franchise and that the site was powered by an IDX solution that was compliant with the local IDX rules - even more so - it lists the listing agent as well as the listing broker office.
Another observation is that the company that provides the site is quite popular with agents for generating leads for the agents. Could it be because the site is indexable, crawlable, and contains relevant, quality content? Sure it is!
Since this is a national franchise site and one of good reputation that was apparently indexable by Scrapers, Scrappers, and Search Engines it occured to me that they may in fact have offices within the MIBOR MLS area. In fact they do! I visited and ironically came across a listing on the MIBOR system that is the listing of the agent that filed the complaint against Paula Henry and Red Door Realty. Here it is - http://www.weichert.com/23260264/?cityid=24765
Why can a National Franchise display indexable MIBOR IDX listings and not Paula Henry?
In one of the many comments from Paula’s posts was one from Paula herself where she stated that the MIBOR cease and desist letter pertained to her and that even though there may be others within MIBOR that had indexable IDX’s, that MIBOR would not be policing the agents websites. Only sites that were turned in by other agents would be affected and served cease and desist letters.
My purpose here is not to rat out the other agents nor the National franchise, but to draw more attention to this issue. The National franchises need to step up and call the NAR immediately getting this simple understandable language put into policy now:
Participants must protect IDX information from unauthorized uses. This requirement does not prohibit indexing of IDX sites by search engines.
National Franchises
I did not point out a particular name in the above paragraphs pertaining to National franchises. I did not think it important. The issue is important. Those of you that looked hard enough though saw that it was Weichert. Weichert is not being ratted out, they are being called into the fight! They are not entitled to anything from MIBOR that Paula Henry is not.
I am calling out for support on this issue from other National franchises as well! You have agents and brokers that are doing a fine job of representing their industry, their franchise, their clients, and at being honest with the public at large whether client, customer, or neither.
These are some of the companies that I know of that have agents, brokers, or the franchises themselves that have websites with indexable IDX’s and this will not be all of them:
- Remax
- Coldwell Banker
- Keller Williams
- Sothebys
- Coldwell Banker
There are also numerous smaller brokerages that do as well. It is time for you all to come out of the woodwork and leave a comment. The NAR will be listening through their Social Media devices!
Comments
6 Responses to “E&0 - Editorial and Opinion”
Got something to say?
Polls
Loading ...
[...] MLS content on your real estate website Google Threatens Real Estate Professionals. Or Does It? MIBOR and NAR policies on IDX Feeds in regard to indexing by Google I know there are more…I’ll keep adding to the list. Thinking I may put these into [...]
[...] Have you examined the irony of the MIBOR and NAR policies on IDX Feeds in regard to indexing by Goog… [...]
I made hundreds of comments on this issue. Not just that one.
A. We didn’t fly Paula and Jay to DC to smooth some feathers. We flew them there to convince the MLS committee that the rule is creating a situation that harms agents. That the rule puts agents at a disadvantage. We also came up with language to enhance the rule. We got that language in front of the committee, along with witness testimony from Paula and Jay, and the committee was convinced to change the rule tight then and there.
The Board of Directors voted to send the issue back to committee based on how quickly these decisions were made. So, I worked to get Jay Thompson added to this very committee to ensure the changes could be examined with the input of someone who was an expert on the reasons why they should be changed.
The issue will hopefully be resolved at the annual convention. That’s what we’re working towards. I’ve published hundreds of comments on this and would be happy to point folks to the exhaustive threads of commentary on this if they like.
I felt compelled to publish this one last comment here as one of my followers on twitter just remarked that I took the easy way out on this issue. There’s a little more to the story @thesachambers.
Wow - I had not idea I would hit a nerve - To Todd - I appreciate your comment above - thank you for the time and effort - the explanation - I had no idea I would be opening up the can of worms I apparently did… and I will continue to do what I can to help with this issue…. I have been working on this for years in my own area…. I apologize if I offended you personally - it is not about you and me - it is about what it right!
The new VOW just ads a new layer of complexity…I am for free exchange of information…In my experience and informed client is a happy client
Any updates on this? I need to know if I have to change all of my RE sites to noindex / nofollow and send all of my web traffic to realtor . com