Initial commit of Ask website

Add ask.openstack.org to openstack-infra. Setup an all-in-one
askbot site based on existing deployment, including apache,
redis,apache solr,postgresql. See askbot.rst for further
details. Refactored to depend on vamsee's puppet solr module.

Depends-On: Iffe07d3a34087cb15151787bc683208425a27594
Change-Id: I36504eac7b953c3cce3e21a3559ac95b1bc12da7

doc/source/askbot.rst

@@ -0,0 +1,119 @@
+:title: Askbot
+
+.. _askbot:
+
+Askbot
+######
+
+Askbot is a publicly available Q&A support site for OpenStack.
+
+At a Glance
+===========
+
+:Hosts:
+  * https://ask.openstack.org
+:Puppet:
+  * :file:`modules/askbot`
+  * :file:`modules/solr`
+  * :file:`modules/openstack_project/manifests/ask.pp`
+:Projects:
+  * https://askbot.com
+  * http://lucene.apache.org/solr
+  * http://redis.io
+
+Overview
+========
+
+The site ask.openstack.org based on the officially released askbot pip distribution.
+The stable deployment is extended with a custom OpenStack theme available at
+https://git.openstack.org/cgit/openstack-infra/askbot-theme.
+
+System Architecture
+===================
+
+::
+
+    +--------+      +----------+
+    | apache | ---- | mod_wsgi |
+    +--------+      +----------+
+                        |
+                 +-------------+    +---------------+
+                 | askbot site |--- | celery daemon |
+                 +-------------+    +---------------+
+                /      |        \
+               /       |         \
+      +-------+  +------------+  +-------------+
+      | redis |  | postgresql |  | apache solr |
+      +-------+  +------------+  +-------------+
+
+Apache / mod_wsgi
+-----------------
+
+Serve the incoming http request using the mod_wsgi Python WSGI adapter, through
+an SSL virtual host. The site vhost also contains url aliases to serve static
+content of the theme and all uploaded image files, including the site logo.
+
+Askbot site
+-----------
+
+The Askbot django application, the custom site specific assets live under
+/srv/askbot-sites/slot0 directory, including the configuration, application
+level log files, static content, custom OpenStack theme and uploaded files.
+
+The authentication based on Google, Yahoo and Launchpad OpenID providers.
+Local login and all other providers except Google, Yahoo and Launchpad are
+disabled in site configuration.
+
+The askbot-theme repository contains just the pure Sass source of the theme,
+so this must be precompiled by compass Sass tool.
+
+Application management tool can be found under /srv/askbot-sites/slot0/config:
+``python manage.py <command>``
+
+Configuration files:
+
+* :file:`modules/askbot/templates/askbot.vhost.erb`
+* :file:`modules/askbot/templates/settings.py.erb`
+
+Celery daemon
+-------------
+
+This upstart based daemon is responsible for async tasks of the Askbot site,
+and can be managed by standard service management tools:
+``server askbot-celeryd <start|stop|status>``
+
+Redis
+-----
+
+Askbot is using redis for handling local caching of configuration and page
+data. It is useful to clear the redis cache with the ``FLUSHALL`` command
+after a service restart.
+
+Postgresql
+----------
+
+A postgresql database hosts the content and dynamic site configuration.
+
+Apache Solr
+-----------
+
+Apache Solr handling the full-text indexes of the site, based on a
+multi-core setup, and assigning cores for specific languages. Currently
+the English (en) and Chinese (zh) languages are supported.
+
+Solr schema templates can be found at:
+
+* :file:`modules/askbot/templates/solr/schema.en.xml.erb`
+* :file:`modules/askbot/templates/solr/schema.cn.xml.erb`
+
+Operational notes
+-----------------
+
+The askbot website contains a ``surprisingly`` askbot based support forum,
+and a lot of operational related information is available there. Additional
+maintenance commands:
+
+* synchronize db schema: ``python manage.py syncdb``
+* migrate database between upgrades: ``python manage.py migrate``
+* rebuild solr index: ``python manage.py askbot_rebuild_index -l <language-code>``
+* assign administrator right to a user: ``python manage.py add_admin <user-id>``

doc/source/systems.rst

@@ -30,6 +30,7 @@ Major Systems
    storyboard
    kerberos
    afs
+   askbot
 
 .. NOTE(dhellmann): These projects were not listed above, or in any
    other toctree, which breaks the build. It's not clear why they were

manifests/site.pp

@@ -702,4 +702,17 @@ node /^afs.*\..*\.openstack\.org$/ {
   }
 }
 
+# Node-OS: precise
+node 'ask.openstack.org' {
+  class { 'openstack_project::ask':
+    sysadmins                    => hiera('sysadmins', []),
+    db_user                      => hiera('ask_db_user', 'ask'),
+    db_password                  => hiera('ask_db_password', 'XXX'),
+    redis_password               => hiera('ask_redis_password', 'XXX'),
+    site_ssl_cert_file_contents  => hiera('ask_site_ssl_cert_file_contents', undef),
+    site_ssl_key_file_contents   => hiera('ask_site_ssl_key_file_contents', undef),
+    site_ssl_chain_file_contents => hiera('ask_site_ssl_chain_file_contents', undef),
+  }
+}
+
 # vim:sw=2:ts=2:expandtab:textwidth=79

modules/openstack_project/manifests/ask.pp

@@ -0,0 +1,138 @@
+# == Class: openstack_project::ask
+#
+# ask.openstack.org Q&A support website
+#
+class openstack_project::ask (
+  $sysadmins = [],
+  $db_name = 'askbotdb',
+  $db_user = undef,
+  $db_password = undef,
+  $slot_name = 'slot0',
+  $redis_enabled = true,
+  $redis_port = '6378',
+  $redis_max_memory = '256m',
+  $redis_bind = '127.0.0.1',
+  $redis_password = undef,
+  $site_ssl_enabled = true,
+  $site_ssl_cert_file_contents = undef,
+  $site_ssl_key_file_contents = undef,
+  $site_ssl_chain_file_contents = undef,
+  $site_ssl_cert_file = '/etc/ssl/certs/ask.openstack.org.pem',
+  $site_ssl_key_file = '/etc/ssl/private/ask.openstack.org.key',
+  $site_ssl_chain_file = '/etc/ssl/certs/ask.openstack.org_ca.pem',
+  $site_name = 'ask.openstack.org',
+  $solr_version = '4.7.2',
+) {
+  class { 'openstack_project::server':
+    iptables_public_tcp_ports => [22, 80, 443],
+    sysadmins                 => $sysadmins,
+  }
+
+  # solr search engine
+  class { 'solr':
+    mirror  => 'http://apache.mesi.com.ar/lucene/solr',
+    version => $solr_version,
+    cores   => [ 'core-default', 'core-en', 'core-zh' ],
+  }
+
+  file { '/usr/share/solr/core-en/conf/schema.xml':
+    ensure  => present,
+    content => template('openstack_project/askbot/schema.en.xml.erb'),
+    replace => true,
+    owner   => 'jetty',
+    group   => 'jetty',
+    mode    => '0644',
+    require => File['/usr/share/solr/core-zh/conf'],
+  }
+
+  file { '/usr/share/solr/core-zh/conf/schema.xml':
+    ensure  => present,
+    content => template('openstack_project/askbot/schema.cn.xml.erb'),
+    replace => true,
+    owner   => 'jetty',
+    group   => 'jetty',
+    mode    => '0644',
+    require => File['/usr/share/solr/core-en/conf'],
+  }
+
+  # deploy smartcn Chinese analyzer from solr contrib/analysys-extras
+  file { "/usr/share/solr/WEB-INF/lib/lucene-analyzers-smartcn-${solr_version}.jar":
+    ensure  => present,
+    replace => 'no',
+    source  => "/tmp/solr-${solr_version}/contrib/analysis-extras/lucene-libs/lucene-analyzers-smartcn-${solr_version}.jar",
+    owner   => 'root',
+    group   => 'root',
+    mode    => '0644',
+    require => Exec['copy-solr'],
+  }
+
+  # postgresql database
+  class { 'postgresql::server': }
+
+  postgresql::server::db { $db_name:
+    user     => $db_user,
+    password => postgresql_password($db_user, $db_password),
+  }
+
+  # redis cache
+  class { 'redis':
+    redis_port       => $redis_port,
+    redis_max_memory => $redis_max_memory,
+    redis_bind       => $redis_bind,
+    redis_password   => $redis_password,
+  }
+
+  # apache http server
+  include apache
+
+  # askbot
+  class { 'askbot':
+    redis_enabled        => $redis_enabled,
+    db_provider          => 'pgsql',
+    require              => Postgresql::Server::Db[$db_name],
+  }
+
+  # custom askbot theme from openstack-infra/askbot-theme repo
+
+  vcsrepo { "/srv/askbot-sites/${slot_name}/themes":
+    ensure   => latest,
+    provider => git,
+    revision => 'master',
+    source   => 'https://git.openstack.org/openstack-infra/askbot-theme',
+    require  => [
+      Class['askbot'], File["/srv/askbot-sites/${slot_name}"],
+      Package['git']
+    ],
+    notify   => [
+      Exec["theme-bundle-install-${slot_name}"],
+      Exec["theme-bundle-compile-${slot_name}"],
+    ],
+  }
+
+  askbot::compass { $slot_name:
+  }
+
+  askbot::site { $site_name:
+    slot_name                    => $slot_name,
+    askbot_debug                 => false,
+    custom_theme_enabled         => true,
+    custom_theme_name            => 'os',
+    redis_enabled                => $redis_enabled,
+    redis_port                   => $redis_port,
+    redis_max_memory             => $redis_max_memory,
+    redis_bind                   => $redis_bind,
+    redis_password               => $redis_password,
+    site_ssl_enabled             => true,
+    site_ssl_cert_file_contents  => $site_ssl_cert_file_contents,
+    site_ssl_key_file_contents   => $site_ssl_key_file_contents,
+    site_ssl_chain_file_contents => $site_ssl_chain_file_contents,
+    site_ssl_cert_file           => $site_ssl_cert_file,
+    site_ssl_key_file            => $site_ssl_key_file,
+    site_ssl_chain_file          => $site_ssl_chain_file,
+    db_provider                  => 'pgsql',
+    db_name                      => $db_name,
+    db_user                      => $db_user,
+    db_password                  => $db_password,
+    require                      => [ Class['redis'], Class['askbot'] ],
+  }
+}

modules/openstack_project/templates/askbot/schema.cn.xml.erb

@@ -0,0 +1,179 @@
+<?xml version="1.0" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<schema name="default" version="1.4">
+  <types>
+    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
+    <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
+    <fieldtype name="binary" class="solr.BinaryField"/>
+
+    <!-- Numeric field types that manipulate the value into
+         a string value that isn't human-readable in its internal form,
+         but with a lexicographic ordering the same as the numeric ordering,
+         so that range queries work correctly. -->
+    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+    <fieldType name="float" class="solr.TrieFloatField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+    <fieldType name="long" class="solr.TrieLongField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+    <fieldType name="double" class="solr.TrieDoubleField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+
+    <fieldType name="tint" class="solr.TrieIntField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tfloat" class="solr.TrieFloatField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tlong" class="solr.TrieLongField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+
+    <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="0" positionIncrementGap="0"/>
+    <!-- A Trie based date field for faster date range queries and date faceting. -->
+    <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
+
+    <fieldType name="point" class="solr.PointType" dimension="2" subFieldSuffix="_d"/>
+    <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/>
+    <fieldtype name="geohash" class="solr.GeoHashField"/>
+
+    <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
+      <analyzer type="index">
+
+        <tokenizer class="solr.SmartChineseSentenceTokenizerFactory"/>
+        <filter class="solr.SmartChineseWordTokenFilterFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <!-- <filter class="solr.PositionFilterFactory" /> -->
+
+      </analyzer>
+      <analyzer type="query">
+
+        <tokenizer class="solr.SmartChineseSentenceTokenizerFactory"/>
+        <filter class="solr.SmartChineseWordTokenFilterFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <!-- <filter class="solr.PositionFilterFactory" /> -->
+
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
+      <analyzer type="index">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.EnglishPossessiveFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+        <!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
+          <filter class="solr.EnglishMinimalStemFilterFactory"/>
+        -->
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.EnglishPossessiveFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+        <!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
+          <filter class="solr.EnglishMinimalStemFilterFactory"/>
+        -->
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
+      <analyzer>
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="ngram" class="solr.TextField" >
+      <analyzer type="index">
+        <tokenizer class="solr.KeywordTokenizerFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.NGramFilterFactory" minGramSize="3" maxGramSize="15" />
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.KeywordTokenizerFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="edge_ngram" class="solr.TextField" positionIncrementGap="1">
+      <analyzer type="index">
+        <tokenizer class="solr.WhitespaceTokenizerFactory" />
+        <filter class="solr.LowerCaseFilterFactory" />
+        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
+        <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15" side="front" />
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.WhitespaceTokenizerFactory" />
+        <filter class="solr.LowerCaseFilterFactory" />
+        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
+      </analyzer>
+    </fieldType>
+  </types>
+
+  <fields>
+   <field name="_version_" type="long" indexed="true" stored="true"/>
+   <field name="_root_" type="string" indexed="true" stored="false"/>
+
+    <!-- general -->
+    <field name="id" type="string" indexed="true" stored="true" multiValued="false" required="true"/>
+    <field name="django_ct" type="string" indexed="true" stored="true" multiValued="false"/>
+    <field name="django_id" type="string" indexed="true" stored="true" multiValued="false"/>
+
+    <dynamicField name="*_i"  type="int"    indexed="true"  stored="true"/>
+    <dynamicField name="*_s"  type="string"  indexed="true"  stored="true"/>
+    <dynamicField name="*_l"  type="long"   indexed="true"  stored="true"/>
+    <dynamicField name="*_t"  type="text_en"    indexed="true"  stored="true"/>
+    <dynamicField name="*_b"  type="boolean" indexed="true"  stored="true"/>
+    <dynamicField name="*_f"  type="float"  indexed="true"  stored="true"/>
+    <dynamicField name="*_d"  type="double" indexed="true"  stored="true"/>
+    <dynamicField name="*_dt" type="date" indexed="true" stored="true"/>
+    <dynamicField name="*_p" type="location" indexed="true" stored="true"/>
+    <dynamicField name="*_coordinate"  type="tdouble" indexed="true"  stored="false"/>
+
+
+    <field name="thread_id" type="long" indexed="true" stored="true" multiValued="false" />
+
+    <field name="author" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+    <field name="text" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+    <field name="post_text" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+    <field name="tags" type="text_en" indexed="true" stored="true" multiValued="true" />
+
+    <field name="title" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+    <!-- required by solr 4 -->
+    <field name="_version_" type="long" indexed="true" stored="true"/>
+
+  </fields>
+
+  <!-- field to use to determine and enforce document uniqueness. -->
+  <uniqueKey>id</uniqueKey>
+
+  <!-- field for the QueryParser to use when an explicit fieldname is absent -->
+  <defaultSearchField>text</defaultSearchField>
+
+  <!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
+  <solrQueryParser defaultOperator="AND"/>
+</schema>

modules/openstack_project/templates/askbot/schema.en.xml.erb

@@ -0,0 +1,170 @@
+<?xml version="1.0" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements.  See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<schema name="default" version="1.4">
+  <types>
+    <fieldtype name="string"  class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
+    <fieldType name="boolean" class="solr.BoolField" sortMissingLast="true" omitNorms="true"/>
+    <fieldtype name="binary" class="solr.BinaryField"/>
+
+    <!-- Numeric field types that manipulate the value into
+         a string value that isn't human-readable in its internal form,
+         but with a lexicographic ordering the same as the numeric ordering,
+         so that range queries work correctly. -->
+    <fieldType name="int" class="solr.TrieIntField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+    <fieldType name="float" class="solr.TrieFloatField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+    <fieldType name="long" class="solr.TrieLongField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+    <fieldType name="double" class="solr.TrieDoubleField" precisionStep="0" omitNorms="true" sortMissingLast="true" positionIncrementGap="0"/>
+
+    <fieldType name="tint" class="solr.TrieIntField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tfloat" class="solr.TrieFloatField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tlong" class="solr.TrieLongField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+    <fieldType name="tdouble" class="solr.TrieDoubleField" precisionStep="8" omitNorms="true" positionIncrementGap="0"/>
+
+    <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="0" positionIncrementGap="0"/>
+    <!-- A Trie based date field for faster date range queries and date faceting. -->
+    <fieldType name="tdate" class="solr.TrieDateField" omitNorms="true" precisionStep="6" positionIncrementGap="0"/>
+
+    <fieldType name="point" class="solr.PointType" dimension="2" subFieldSuffix="_d"/>
+    <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/>
+    <fieldtype name="geohash" class="solr.GeoHashField"/>
+
+    <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
+      <analyzer type="index">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="text_en" class="solr.TextField" positionIncrementGap="100">
+      <analyzer type="index">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.EnglishPossessiveFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+        <!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
+          <filter class="solr.EnglishMinimalStemFilterFactory"/>
+        -->
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.StandardTokenizerFactory"/>
+        <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
+        <filter class="solr.StopFilterFactory"
+                ignoreCase="true"
+                words="stopwords.txt"
+                enablePositionIncrements="true"
+                />
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.EnglishPossessiveFilterFactory"/>
+        <filter class="solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
+        <!-- Optionally you may want to use this less aggressive stemmer instead of PorterStemFilterFactory:
+          <filter class="solr.EnglishMinimalStemFilterFactory"/>
+        -->
+        <filter class="solr.PorterStemFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="text_ws" class="solr.TextField" positionIncrementGap="100">
+      <analyzer>
+        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="ngram" class="solr.TextField" >
+      <analyzer type="index">
+        <tokenizer class="solr.KeywordTokenizerFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+        <filter class="solr.NGramFilterFactory" minGramSize="3" maxGramSize="15" />
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.KeywordTokenizerFactory"/>
+        <filter class="solr.LowerCaseFilterFactory"/>
+      </analyzer>
+    </fieldType>
+
+    <fieldType name="edge_ngram" class="solr.TextField" positionIncrementGap="1">
+      <analyzer type="index">
+        <tokenizer class="solr.WhitespaceTokenizerFactory" />
+        <filter class="solr.LowerCaseFilterFactory" />
+        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
+        <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="15" side="front" />
+      </analyzer>
+      <analyzer type="query">
+        <tokenizer class="solr.WhitespaceTokenizerFactory" />
+        <filter class="solr.LowerCaseFilterFactory" />
+        <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
+      </analyzer>
+    </fieldType>
+  </types>
+
+  <fields>
+   <field name="_version_" type="long" indexed="true" stored="true"/>
+   <field name="_root_" type="string" indexed="true" stored="false"/>
+
+    <!-- general -->
+    <field name="id" type="string" indexed="true" stored="true" multiValued="false" required="true"/>
+    <field name="django_ct" type="string" indexed="true" stored="true" multiValued="false"/>
+    <field name="django_id" type="string" indexed="true" stored="true" multiValued="false"/>
+
+    <dynamicField name="*_i"  type="int"    indexed="true"  stored="true"/>
+    <dynamicField name="*_s"  type="string"  indexed="true"  stored="true"/>
+    <dynamicField name="*_l"  type="long"   indexed="true"  stored="true"/>
+    <dynamicField name="*_t"  type="text_en"    indexed="true"  stored="true"/>
+    <dynamicField name="*_b"  type="boolean" indexed="true"  stored="true"/>
+    <dynamicField name="*_f"  type="float"  indexed="true"  stored="true"/>
+    <dynamicField name="*_d"  type="double" indexed="true"  stored="true"/>
+    <dynamicField name="*_dt" type="date" indexed="true" stored="true"/>
+    <dynamicField name="*_p" type="location" indexed="true" stored="true"/>
+    <dynamicField name="*_coordinate"  type="tdouble" indexed="true"  stored="false"/>
+
+
+    <field name="thread_id" type="long" indexed="true" stored="true" multiValued="false" />
+
+    <field name="author" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+    <field name="text" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+    <field name="post_text" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+    <field name="tags" type="text_en" indexed="true" stored="true" multiValued="true" />
+
+    <field name="title" type="text_en" indexed="true" stored="true" multiValued="false" />
+
+  </fields>
+
+  <!-- field to use to determine and enforce document uniqueness. -->
+  <uniqueKey>id</uniqueKey>
+
+  <!-- field for the QueryParser to use when an explicit fieldname is absent -->
+  <defaultSearchField>text</defaultSearchField>
+
+  <!-- SolrQueryParser configuration: defaultOperator="AND|OR" -->
+  <solrQueryParser defaultOperator="AND"/>
+</schema>