Designing a social discovery Engine part I
Hi, I am Sumin Sudhi, Sr. PHP Developer at Globussoft with about 3 years of work experience across various platforms. Of late, I have been working on Frompo – a social discovery engine. With my experience, I am going to tell you what’s a social discovery engine and how to go about designing one.
I know that most of our users are well aware of the term Social Network and are part of one of many social networks available in internet today. Some of those are Facebook, twitter, LinkedIn, Google+.
But the topic which I am most concerned about is a Social Discovery Engine.
So what exactly is a social discovery engine?How does it help to hack our life and makes thing much easier? Why is it worth so much?These may be one the few questions that may be in your mind right now.
What is a Social Discovery Engine?
Social discovery is the practice of utilizing technology to find and meet new people. Technology is used to discover new people and sometimes new experiences shopping, meeting friends or even traveling. The discovery of new people is often in real-time, enabled by mobile apps. However, social discovery is not limited to meeting people in real-time; it also leads to sales and revenue for companies via social media.
An example of retail would be the addition of social sharing with music, through the iTunes music store. There is a social component to discovering new music.
Evolution of Social Discovery Engine
Think in a way you all that do in a social network post, like, share, comment, create new friends, messaging.
Now suppose you do search something as you do in a search engine and you can get everything related to that keyword like a 360 degree view around it including the comments of your friends and their friends global views about some place things or object associated to the keyword wouldn’t it be awesome.
The search is usually not restricted to a domain, it is rather a combination of vertical search engines giving a view of multi search engine + social features
Features
Some features that you would likely to see in a social discovery engine are:
- • Keep yourself updated with happenings
- • Results from different niches in the same surface
- • Share your ideas and thought and contribute to knowledge bases
- • Get to know more about anything based upon your search habits and relevancy options
- • Manage multiple social networks
- • Schedule messages and tweets
- • Track brand mentions
- • Analyze social media traffic
Ideas of Development
Beginning with the development we have to keep some points in mind:
• Database Selection and Design for Scalable Applications: The success of a heavy load application is mostly based upon its database architecture. Improper design may lead to application downfall. Some of the most essential points involve:-
- o Selection of database engine depending upon its scope
- o Designing a database architecture that can serve the purpose
- o Evolving and optimizing it based on normalization and load factors
- o Load testing it to get the most desirable output within the shortest possible time .This would help to analyze the slow queries and improve efficiency of engine.
• Web scraping and indexing algorithms: Scraping is a process of scanning the web for obtaining the results. Depending upon the user search the engine should be smart enough to get all the relevant data from the web and show under one hood.
- o Creating a web spider or use existing ones? Learning web automation.
- o Understanding Lucene based architecture and implementing one.
- o Think of web spiders and robots that will be curious to know what’s on your site.
- o SEO and proper impression of your contents.
• User Authentication Privacy and Search preference Algorithms: Authenticating persons on such an engine would be little complex in comparison to a traditional system. As in this case the user would be able to login with their favorite social network engine, so the engine must be able not only to verify user identity and also protect it from identity thefts.
The SDE (Social Discovery Engine) would not only evolve just as data aggregator but also be an improvement over any rank/relevancy based system by understanding and evolving based on user search preferences and habits.
• Tracking web traffic and its analysis: Tracking web traffic is one of the most essential parts that would help you to understand the success or failure of your system. This work in turn helps you to decide over the areas that you could work on to improve the SDE to whole new level.
