What is KindleEar?

KindleEar is a free webapp that running on Google App Engine, it aggregates feeds and deliver the generated docs to your Kindle periodically.

Features:

Benefits:

Dows this application require its own server?

This application is hosted on Google App Engine (GAE), GAE has free quotas enough for small applications like KindleEar, you don't need to pay a penny to run KindleEar in GAE, it's fine if you don't own a server(VPS or other infrastructure).

Remember: GAE(Google App Engine) is not GCE (Google Computer Engine), GAE is free forever (with limit of quotas of resources), GCE is free only within 60 days trial. you don't need to start a GCE trial.

How to deploy my own delivery server?

The formal way:

  1. Download the latest version of KindleEar on github and then extract it to somewhere, for example C:\KindleEar (we'll use it in the following steps).
  2. You need a google account, please visit the Google Developers Console page. Use your gmail account to login, click on "Create an empty Project", follow steps to create a GAE application.
  3. Install Python 2.7.x. Download the latest 2.7.x source release and then install it. Do not install a Python 3.x version, because GAE does not support it now.
  4. Install GAE SDK, which will be used to upload a GAE program. In section on this page, click "Google App Engine SDK for Python", choose your platform, click to download the corresponding version, then install, all the way "next" without words.
  5. Go to the previously downloaded KindleEar directory (e.g. C:\KindleEar), use a text editor to open app.yaml and module-worker.yaml file, modify the word 'kindleear' in the first line to your GAE application name (e.g. kindleear123), then save and exit.
  6. Modify variables 'SRC_EMAIL' and 'DOMAIN' and 'TIMEZONE' in config.py in same directory of app.yaml. You can modify other variables if you want and understand what it means.Using the default values are recommended.
  7. Start upload KindleEar now. You can use Google App Engine Launcher GUI or command line. Because Launcher is not stable, we'll use the command line in Windows.
  8. After successfully uploading KindleEar, use your browser to open the site: http://kindleear123.appspot.com (example only, kindleear123 should be changed to the name of your GAE application). If everything goes fine, then go to the last step.
  9. Add your gmail account to your kindle email whitelist. You should know how to do it already.
  10. Then what? Enjoy your own delivery service now!
  11. Tips: the initial username of application is admin, default password is admin too.

The easy way(recommended):

  1. Download KindleEar and extract it, get a GAE account, create an application.
  2. Download uploader. (for windows user only)
  3. Extract 'uploader' to a directory, move the folder of KindleEar into the folder of 'uploader' and rename it to kindleear, then double-click 'uploader.bat' to upload.
  4. Tips: the initial username of application is admin, default password is admin too.

Why do I not receive the deliveries?

There are many possible reasons:

  1. You haven't enabled the "Enable deliver custom rss" option, this option is at the bottom of the setting page.
  2. You haven't added your gmail address to Amazon's 'Approved Personal Document E-mail List'.
  3. The content of the rss link isn't a valid XML format.
  4. For those feeds generated by Feedburner, you can append a query string '?format=xml' to end of url, for example: http://xx.com/feed?format=xml.

What is full text rss?

Full text rss means that it contains all the contents of the articles in the rss, while summary rss contains only brief descriptions of the articles. You can check it by opening the link of the rss.

Can full text rss be processed as summary rss? What about the reverse?

Full text rss can be treated as summary rss. In this case, program will ignore the content in the rss xml file and fetch full text from the original link. It costs much time which will result the decreasing of supported rss quantities. If summary rss is processed as full text rss, articles sent to your kindle would still have brief descriptions only.

How to deal with weekly subscription?

Some rss are updated weekly. For this type of rss, you can touch a new python file in the 'books' directory with variable 'deliver_days' setted to a day.
deliver_days=['Friday']

Can KindleEar deliver news to my kindle more than once a day?

The answer is yes, follow the next steps to make it come true:

  1. Add a python file to the directory of 'books'.
  2. Set 'deliver_times' using a list in the new python file.
    For example:
    deliver_times=[6,14,22] #6:00,14:00,22:00 deliver three times a day
  3. Optional: set 'oldest_article' to the number of interval seconds between two deliveries.
    For example:
    oldest_article=28800 #8*60*60

How to use the url filter?

Url filter is not for beginners, you need to know regular expression and python. Urls in the list would not be downloaded for saving time.

What can I do if the gfw of China blocked some rss?

Change the link to use https protocol.

How to put the cover or the masthead images in the same directory when I add a customize book?

Set the variable mastheadfile and coverfile to a path, for example:
mastheadfile = 'books/xxxx.gif'
coverfile = 'books/xxxx.jpg'

How to deal with the SSL error problem when I upload KindleEar?

Some friends in China would encounter "urllib2.URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol>". To solve this problem, try to add a new line "74.125.229.174 appengine.google.com" to the hosts file.

Why some articles have garbled characters?

For custom rss and the built-in books without encoding specified, program detects encoding automatically by using http header, html meta or the chardet module of python. Howerver, the result is not 100% accurate. If the encoding detecting is wrong, the article may be garbled. Because this situation is very rare, KindleEar only supplies an not very elegant solution: manually change the chardet-detect encoding saved in the database. Steps as follows:

  1. Login appid.appspot.com by using account 'admin'(assuming your application name is appid).
  2. Visit 'https://appid.appspot.com/dbviewer'. Check the encoding of the website whose articles are garbled in the UrlEncoding section. If the encoding is wrong, take down its ID.
  3. Now visit 'https://appid.appspot.com/dbviewer?action=modurlenc&id=dbid&feedenc=fenc&pageenc=penc'
  4. Or you can change them in 'Datastore viewer' in the GAE backend.
  5. You can get the correct encoding by reading source code of a webpage.
  6. If you cannot find the website in page 'dbviewer', it means KindleEar has never detected the encoding of any page of this website, http header or html meta is used only. In this case, you can set the variable 'ALWAYS_CHAR_DETECT' in 'config.py' to True.
  7. If the article still has garbled characters after setting 'ALWAYS_CHAR_DETECT' to True, it means that encoding detected by chardet is wrong and doesn't match with the http header or the html meta. The final attempt is to set 'TRUST_ENCODING_IN_HEADER_OR_META' to True.
  8. With all the solutions tested and the problem still exists, please add the rss to a python file in the folder 'books' with encoding specified.

What can I do if I forget the password?

If you are a non-administrator user, and you forget the password, you can ask the administrator to change your password. If the administrator password was forgot, you can login appengine.google.com, select "Datastore Admin" on the left, delete all KeUser data, all the account information will be deleted. And then re-use admin/admin to login in your application . Did not find "Datastore Admin"? You can activate it in the "Application Settings".
If you know python, I'm sure that you can find other solutions.

How to use mail service xxx@appid.appspotmail.com?

You'll have uncountable mail addresses like xxx@appid.appspotmail.com as long as xxx is valid ascii string after you have deployed KindleEar(appid is your application id).

Why I can't lookup words in the delivery books after I chosen 'Merge books into one'? 

With the option 'Merge books in one' checked, KindleEar will set the metadata of the book as Custom Rss. To fix this problem, you have to set the language attribute in the 'Custom Rss Setting' section manually.

How to deal with those websites that need login before I can view the articles?

If you want to crawl a website that needs login, you have to add a python file to the folder 'books'. Refer to sample books if you don't know how to write it.

  1. needs_subscription : Set to True
  2. login_url : The URL for login page
  3. form_4_login : Optional, if not set, KindleEar will guess which form in the page is used to login. Set to an integer indicate the sequence number (where 0 is the first) of the form in the page. Set to a string indicate the name of the form, you can use CSS selector: '#' for id and '.' for class, string only will match any of it(name/id/class).
  4. Then login to KindleEar. In the 'Feeds' page, a new 'Login info' button will appear on the right side of the book which needs_subscription setted to True in the 'Subscripted' section. Click on it and then fill in your account name and password.

Note1: currently we don't support those sites that need run javascript to login or need to input a captcha.

Note2: for some special site, you may have to overwirte the 'login' function which inherits from the 'BaseFeedBook' class.

Note3: the password will be stored encrypted on GAE, the key for encryption is a random string for each account of KindleEar, but the key is also stored on GAE, so it's not absolutely safe, please keep this in mind.

Can I delete/hide books in Chinese?

If you are not Chinese, you can delete some *.py files in directory 'books'.
'__init__.py' and 'base.py' are required, please don't remove them.
If you deploy the application via uploader, uploader can help you to delete books which language is Chinese.

What's the meaning of the "Separate" option for the built-in books?

If you choose this option, the book you chosen will be sent to kindle separately even if the option "Merge books into one" is chosen.

How to solve it if website shows "internal server error" exception.

Open appengine.google.com, choose your application, open "Logs", if have “NeedIndexError: no matching index found.” exception, then you can execute command in your computer.
python appcfg.py update_indexes YourKindleEarFolder/
if have “UnknownQueueError” exception, execute:
python appcfg.py update_queues YourKindleEarFolder/
After command be executed, please wait some miniutes for GAE generate index for you. If the problem still exist, you should deploy KindleEar again.

How to use the feature 'Save to Pocket'?

The Pocket is a popular 'Read It Later' service. The service allows the user to save an article or web page to the cloud for later reading. The article is then sent to the user's Pocket list (synced to all of their devices) for offline reading. Pocket removes clutter from articles and allows the user to adjust text settings for easier reading.
Now you can save an article of your Kindle to your pocket's account, and then you can read it in other devices later.

Follows steps to enjoy this feature:

Remarks:

My logs shows error 'wrong SRC_EMAIL'?

Go to your GAE console, in 'Settings' | 'Application settings' page, add your email into field 'Email API authorized senders'.