lsm/Orchard
1
0
Fork 0
mirror of https://github.com/OrchardCMS/Orchard.git synced 2025-11-08 18:44:58 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Created indexing/search default implementation with Lucene

Browse Source 
IIndexManager to handle the available IIndexProvider instances
ISearchBuilder created from the used IIndexProvider to create search queries
Indexing background task records
Background service to execute indexing tasks
Unit tests for indexing and searching

--HG--
branch : dev
This commit is contained in:
Sebastien Ros
2010-06-02 15:56:54 -07:00
parent 3064a5fcb7
commit 015a0503bb
37 changed files with 28863 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

239
lib/lucene.net/LICENSE.txt Normal file
View File

@@ -0,0 +1,239 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Some code in src/java/org/apache/lucene/util/UnicodeUtil.java was
derived from unicode conversion examples available at
http://www.unicode.org/Public/PROGRAMS/CVTUTF. Here is the copyright
from those sources:
/*
* Copyright 2001-2004 Unicode, Inc.
*
* Disclaimer
*
* This source code is provided as is by Unicode, Inc. No claims are
* made as to fitness for any particular purpose. No warranties of any
* kind are expressed or implied. The recipient agrees to determine
* applicability of information provided. If this file has been
* purchased on magnetic or optical media from Unicode, Inc., the
* sole remedy for any claim will be exchange of defective media
* within 90 days of receipt.
*
* Limitations on Rights to Redistribute This Code
*
* Unicode, Inc. hereby grants the right to freely use the information
* supplied in this file in the creation of products supporting the
* Unicode Standard, and to make copies of this file in any form
* for internal or external distribution as long as this notice
* remains attached.
*/
Some code in src/java/org/apache/lucene/util/ArrayUtil.java was
derived from Python 2.4.2 sources available at
http://www.python.org. Full license is here:
http://www.python.org/download/releases/2.4.2/license/

27275
lib/lucene.net/Lucene.Net.xml Normal file
View File

File diff suppressed because it is too large Load Diff

146
src/Orchard.Core.Tests/Indexing/DefaultIndexProviderTests.cs Normal file
View File

@@ -0,0 +1,146 @@
using System;
using System.IO;
using System.Linq;
using Autofac;
using NUnit.Framework;
using Orchard.Environment.Configuration;
using Orchard.FileSystems.AppData;
using Orchard.Indexing;
using Orchard.Core.Indexing.Lucene;
namespace Orchard.Tests.Indexing {
public class DefaultIndexProviderTests {
private IContainer _container;
private IIndexProvider _provider;
private IAppDataFolder _appDataFolder;
private ShellSettings _shellSettings;
private readonly string _basePath = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
[TestFixtureTearDown]
public void Clean() {
Directory.Delete(_basePath, true);
}
[SetUp]
public void Setup() {
if (Directory.Exists(_basePath)) {
Directory.Delete(_basePath, true);
}
Directory.CreateDirectory(_basePath);
_appDataFolder = new AppDataFolder();
_appDataFolder.SetBasePath(_basePath);
var builder = new ContainerBuilder();
builder.RegisterType<DefaultIndexProvider>().As<IIndexProvider>();
builder.RegisterInstance(_appDataFolder).As<IAppDataFolder>();
// setting up a ShellSettings instance
_shellSettings = new ShellSettings { Name = "My Site" };
builder.RegisterInstance(_shellSettings).As<ShellSettings>();
_container = builder.Build();
_provider = _container.Resolve<IIndexProvider>();
}
private string[] Indexes() {
return new DirectoryInfo(Path.Combine(_basePath, "Sites", "My Site", "Indexes")).GetDirectories().Select(d => d.Name).ToArray();
}
[Test]
public void IndexProviderShouldCreateNewIndex() {
Assert.That(Indexes().Length, Is.EqualTo(0));
_provider.CreateIndex("default");
Assert.That(Indexes().Length, Is.EqualTo(1));
}
[Test]
public void IndexProviderShouldOverwriteAlreadyExistingIndex() {
_provider.CreateIndex("default");
_provider.CreateIndex("default");
}
[Test]
public void IndexProviderShouldDeleteExistingIndex() {
Assert.That(Indexes().Length, Is.EqualTo(0));
_provider.CreateIndex("default");
Assert.That(Indexes().Length, Is.EqualTo(1));
_provider.DeleteIndex("default");
Assert.That(Indexes().Length, Is.EqualTo(0));
}
[Test]
public void IndexProviderShouldListExistingIndexes() {
Assert.That(Indexes().Length, Is.EqualTo(0));
_provider.CreateIndex("default");
Assert.That(Indexes().Length, Is.EqualTo(1));
Assert.That(Indexes()[0], Is.EqualTo("default"));
_provider.CreateIndex("foo");
Assert.That(Indexes().Length, Is.EqualTo(2));
}
[Test]
public void ANewIndexShouldBeEmpty() {
_provider.CreateIndex("default");
var searchBuilder = _provider.CreateSearchBuilder("default");
var hits = searchBuilder.Search();
Assert.That(hits.Count(), Is.EqualTo(0));
}
[Test]
public void DocumentsShouldBeSearchableById() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(42));
var searchBuilder = _provider.CreateSearchBuilder("default");
var hit = searchBuilder.Get(42);
Assert.IsNotNull(hit);
Assert.That(hit.Id, Is.EqualTo(42));
hit = searchBuilder.Get(1);
Assert.IsNull(hit);
}
[Test]
public void PropertiesShouldNotBeLost() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(42).Add("prop1", "value1"));
var hit = _provider.CreateSearchBuilder("default").Get(42);
Assert.IsNotNull(hit);
Assert.That(hit.Id, Is.EqualTo(42));
Assert.That(hit.GetString("prop1"), Is.EqualTo("value1"));
}
[Test]
public void ShouldHandleMultipleIndexes() {
_provider.CreateIndex("default1");
_provider.Store("default1", _provider.New(1));
_provider.CreateIndex("default2");
_provider.Store("default2", _provider.New(2));
_provider.CreateIndex("default3");
_provider.Store("default3", _provider.New(3));
Assert.IsNotNull(_provider.CreateSearchBuilder("default1").Get(1));
Assert.IsNotNull(_provider.CreateSearchBuilder("default2").Get(2));
Assert.IsNotNull(_provider.CreateSearchBuilder("default3").Get(3));
Assert.IsNull(_provider.CreateSearchBuilder("default1").Get(2));
Assert.IsNull(_provider.CreateSearchBuilder("default2").Get(3));
Assert.IsNull(_provider.CreateSearchBuilder("default3").Get(1));
}
}
}

181
src/Orchard.Core.Tests/Indexing/DefaultSearchBuilderTests.cs Normal file
View File

@@ -0,0 +1,181 @@
using System;
using System.IO;
using System.Linq;
using Autofac;
using NUnit.Framework;
using Orchard.Environment.Configuration;
using Orchard.FileSystems.AppData;
using Orchard.Indexing;
using Orchard.Core.Indexing.Lucene;
namespace Orchard.Tests.Indexing {
public class DefaultSearchBuilderTests {
private IContainer _container;
private IIndexProvider _provider;
private IAppDataFolder _appDataFolder;
private ShellSettings _shellSettings;
private readonly string _basePath = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName());
[TestFixtureTearDown]
public void Clean() {
Directory.Delete(_basePath, true);
}
[SetUp]
public void Setup() {
if (Directory.Exists(_basePath)) {
Directory.Delete(_basePath, true);
}
Directory.CreateDirectory(_basePath);
_appDataFolder = new AppDataFolder();
_appDataFolder.SetBasePath(_basePath);
var builder = new ContainerBuilder();
builder.RegisterType<DefaultIndexProvider>().As<IIndexProvider>();
builder.RegisterInstance(_appDataFolder).As<IAppDataFolder>();
// setting up a ShellSettings instance
_shellSettings = new ShellSettings { Name = "My Site" };
builder.RegisterInstance(_shellSettings).As<ShellSettings>();
_container = builder.Build();
_provider = _container.Resolve<IIndexProvider>();
}
private ISearchBuilder _searchBuilder { get { return _provider.CreateSearchBuilder("default"); } }
[Test]
public void SearchTermsShouldBeFoundInMultipleFields() {
_provider.CreateIndex("default");
_provider.Store("default",
_provider.New(42)
.Add("title", "title1 title2 title3")
.Add("date", new DateTime(2010, 05, 28, 14, 13, 56, 123))
);
Assert.IsNotNull(_provider.CreateSearchBuilder("default").Get(42));
Assert.IsNotNull(_provider.CreateSearchBuilder("default").WithField("title", "title1").Search().FirstOrDefault());
Assert.IsNotNull(_provider.CreateSearchBuilder("default").WithField("title", "title2").Search().FirstOrDefault());
Assert.IsNotNull(_provider.CreateSearchBuilder("default").WithField("title", "title3").Search().FirstOrDefault());
Assert.IsNull(_provider.CreateSearchBuilder("default").WithField("title", "title4").Search().FirstOrDefault());
Assert.IsNotNull(_provider.CreateSearchBuilder("default").WithField("title", "title").Search().FirstOrDefault());
}
[Test]
public void ShouldSearchById() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(1));
_provider.Store("default", _provider.New(2));
_provider.Store("default", _provider.New(3));
Assert.That(_searchBuilder.Get(1).Id, Is.EqualTo(1));
Assert.That(_searchBuilder.Get(2).Id, Is.EqualTo(2));
Assert.That(_searchBuilder.Get(3).Id, Is.EqualTo(3));
}
[Test]
public void ShouldSearchWithField() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(1).Add("title", "cat"));
_provider.Store("default", _provider.New(2).Add("title", "dog"));
_provider.Store("default", _provider.New(3).Add("title", "cat"));
Assert.That(_searchBuilder.WithField("title", "cat").Search().Count(), Is.EqualTo(2));
Assert.That(_searchBuilder.WithField("title", "cat").Search().Any(hit => new[] { 1, 3 }.Contains(hit.Id)), Is.True);
}
[Test]
public void ShouldCountResultsOnly() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(1).Add("title", "cat"));
_provider.Store("default", _provider.New(2).Add("title", "dog"));
_provider.Store("default", _provider.New(3).Add("title", "cat"));
Assert.That(_searchBuilder.WithField("title", "dog").Count(), Is.EqualTo(1));
Assert.That(_searchBuilder.WithField("title", "cat").Count(), Is.EqualTo(2));
Assert.That(_searchBuilder.WithField("title", "c").Count(), Is.EqualTo(2));
}
[Test]
public void ShouldFilterByDate() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(1).Add("date", new DateTime(2010, 05, 28, 12, 30, 15)));
_provider.Store("default", _provider.New(2).Add("date", new DateTime(2010, 05, 28, 12, 30, 30)));
_provider.Store("default", _provider.New(3).Add("date", new DateTime(2010, 05, 28, 12, 30, 45)));
Assert.That(_searchBuilder.After("date", new DateTime(2010, 05, 28, 12, 30, 15)).Count(), Is.EqualTo(3));
Assert.That(_searchBuilder.Before("date", new DateTime(2010, 05, 28, 12, 30, 45)).Count(), Is.EqualTo(3));
Assert.That(_searchBuilder.After("date", new DateTime(2010, 05, 28, 12, 30, 15)).Before("date", new DateTime(2010, 05, 28, 12, 30, 45)).Count(), Is.EqualTo(3));
Assert.That(_searchBuilder.After("date", new DateTime(2010, 05, 28, 12, 30, 16)).Before("date", new DateTime(2010, 05, 28, 12, 30, 44)).Count(), Is.EqualTo(1));
Assert.That(_searchBuilder.After("date", new DateTime(2010, 05, 28, 12, 30, 46)).Count(), Is.EqualTo(0));
Assert.That(_searchBuilder.Before("date", new DateTime(2010, 05, 28, 12, 30, 1)).Count(), Is.EqualTo(0));
}
[Test]
public void ShouldSliceResults() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(1));
_provider.Store("default", _provider.New(22));
_provider.Store("default", _provider.New(333));
_provider.Store("default", _provider.New(4444));
_provider.Store("default", _provider.New(55555));
Assert.That(_searchBuilder.Count(), Is.EqualTo(5));
Assert.That(_searchBuilder.Slice(0, 3).Count(), Is.EqualTo(3));
Assert.That(_searchBuilder.Slice(1, 3).Count(), Is.EqualTo(3));
Assert.That(_searchBuilder.Slice(3, 3).Count(), Is.EqualTo(2));
// Count() and Search() should return the same results
Assert.That(_searchBuilder.Search().Count(), Is.EqualTo(5));
Assert.That(_searchBuilder.Slice(0, 3).Search().Count(), Is.EqualTo(3));
Assert.That(_searchBuilder.Slice(1, 3).Search().Count(), Is.EqualTo(3));
Assert.That(_searchBuilder.Slice(3, 3).Search().Count(), Is.EqualTo(2));
}
[Test]
public void ShouldSortByRelevance() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(1).Add("body", "michaelson is in the kitchen"));
_provider.Store("default", _provider.New(2).Add("body", "michael as a cousin named michael"));
_provider.Store("default", _provider.New(3).Add("body", "speak inside the mic"));
_provider.Store("default", _provider.New(4).Add("body", "a dog is pursuing a cat"));
_provider.Store("default", _provider.New(5).Add("body", "the elephant can't catch up the dog"));
var michael = _searchBuilder.WithField("body", "mic").Search().ToList();
Assert.That(michael.Count(), Is.EqualTo(3));
Assert.That(michael[0].Score >= michael[1].Score, Is.True);
// Sorting on score is always descending
michael = _searchBuilder.WithField("body", "mic").Ascending().Search().ToList();
Assert.That(michael.Count(), Is.EqualTo(3));
Assert.That(michael[0].Score >= michael[1].Score, Is.True);
}
[Test]
public void ShouldSortByDate() {
_provider.CreateIndex("default");
_provider.Store("default", _provider.New(1).Add("date", new DateTime(2010, 05, 28, 12, 30, 15)));
_provider.Store("default", _provider.New(2).Add("date", new DateTime(2010, 05, 28, 12, 30, 30)));
_provider.Store("default", _provider.New(3).Add("date", new DateTime(2010, 05, 28, 12, 30, 45)));
var date = _searchBuilder.SortBy("date").Search().ToList();
Assert.That(date.Count(), Is.EqualTo(3));
Assert.That(date[0].GetDateTime("date") > date[1].GetDateTime("date"), Is.True);
Assert.That(date[1].GetDateTime("date") > date[2].GetDateTime("date"), Is.True);
date = _searchBuilder.SortBy("date").Ascending().Search().ToList();
Assert.That(date.Count(), Is.EqualTo(3));
Assert.That(date[0].GetDateTime("date") < date[1].GetDateTime("date"), Is.True);
Assert.That(date[1].GetDateTime("date") < date[2].GetDateTime("date"), Is.True);
}
}
}

2
src/Orchard.Core.Tests/Orchard.Core.Tests.csproj
View File

@@ -104,6 +104,8 @@
<Compile Include="Common\Providers\CommonAspectProviderTests.cs" /> <Compile Include="Common\Providers\CommonAspectProviderTests.cs" />
<Compile Include="Common\Services\RoutableServiceTests.cs" /> <Compile Include="Common\Services\RoutableServiceTests.cs" />
<Compile Include="Feeds\Controllers\FeedControllerTests.cs" /> <Compile Include="Feeds\Controllers\FeedControllerTests.cs" />
<Compile Include="Indexing\DefaultIndexProviderTests.cs" />
<Compile Include="Indexing\DefaultSearchBuilderTests.cs" />
<Compile Include="Properties\AssemblyInfo.cs" /> <Compile Include="Properties\AssemblyInfo.cs" />
<Compile Include="Scheduling\ScheduledTaskManagerTests.cs" /> <Compile Include="Scheduling\ScheduledTaskManagerTests.cs" />
<Compile Include="Scheduling\ScheduledTaskExecutorTests.cs" /> <Compile Include="Scheduling\ScheduledTaskExecutorTests.cs" />

4
src/Orchard.Web/Core/Common/Handlers/BodyAspectHandler.cs
View File

@@ -8,6 +8,10 @@ namespace Orchard.Core.Common.Handlers {
public class BodyAspectHandler : ContentHandler { public class BodyAspectHandler : ContentHandler {
public BodyAspectHandler(IRepository<BodyRecord> bodyRepository) { public BodyAspectHandler(IRepository<BodyRecord> bodyRepository) {
Filters.Add(StorageFilter.For(bodyRepository)); Filters.Add(StorageFilter.For(bodyRepository));
OnIndexing<BodyAspect>((context, bodyAspect) => {
context.IndexDocument.Add("body", bodyAspect.Record.Text);
});
} }
} }
} }

101
src/Orchard.Web/Core/Indexing/Lucene/DefaultIndexDocument.cs Normal file
View File

@@ -0,0 +1,101 @@
using System;
using System.Collections.Generic;
using Lucene.Net.Documents;
using Orchard.Indexing;
namespace Orchard.Core.Indexing.Lucene {
public class DefaultIndexDocument : IIndexDocument {
public List<AbstractField> Fields { get; private set; }
private AbstractField _previousField;
public int Id { get; private set; }
public DefaultIndexDocument(int documentId) {
Fields = new List<AbstractField>();
SetContentItemId(documentId);
}
public IIndexDocument Add(string name, string value) {
AppendPreviousField();
_previousField = new Field(name, value, Field.Store.YES, Field.Index.ANALYZED);
return this;
}
public IIndexDocument Add(string name, DateTime value) {
AppendPreviousField();
_previousField = new Field(name, DateTools.DateToString(value, DateTools.Resolution.SECOND), Field.Store.YES, Field.Index.NOT_ANALYZED);
return this;
}
public IIndexDocument Add(string name, int value) {
AppendPreviousField();
_previousField = new NumericField(name, Field.Store.YES, true).SetIntValue(value);
return this;
}
public IIndexDocument Add(string name, bool value) {
AppendPreviousField();
_previousField = new Field(name, value.ToString().ToLower(), Field.Store.YES, Field.Index.NOT_ANALYZED);
return this;
}
public IIndexDocument Add(string name, float value) {
AppendPreviousField();
_previousField = new NumericField(name, Field.Store.YES, true).SetFloatValue(value);
return this;
}
public IIndexDocument Add(string name, object value) {
AppendPreviousField();
_previousField = new Field(name, value.ToString(), Field.Store.NO, Field.Index.NOT_ANALYZED);
return this;
}
public IIndexDocument Store(bool store) {
EnsurePreviousField();
if(store != _previousField.IsStored()) {
var index = _previousField.IsTokenized() ? Field.Index.ANALYZED : Field.Index.NOT_ANALYZED;
_previousField = new Field(_previousField.Name(), _previousField.StringValue(), store ? Field.Store.YES : Field.Store.NO, index);
}
return this;
}
public IIndexDocument Analyze(bool analyze) {
EnsurePreviousField();
if (_previousField.IsTokenized() == analyze) {
return this;
}
var index = analyze ? Field.Index.ANALYZED : Field.Index.NOT_ANALYZED;
var store = _previousField.IsStored() ? Field.Store.YES : Field.Store.NO;
_previousField = new Field(_previousField.Name(), _previousField.StringValue(), store, index);
return this;
}
public IIndexDocument SetContentItemId(int id) {
Id = id;
Fields.Add(new Field("id", id.ToString(), Field.Store.YES, Field.Index.NOT_ANALYZED));
return this;
}
private void AppendPreviousField() {
if (_previousField == null) {
return;
}
Fields.Add(_previousField);
_previousField = null;
}
public void PrepareForIndexing() {
AppendPreviousField();
}
private void EnsurePreviousField() {
if(_previousField == null) {
throw new ApplicationException("Operation can't be applied in this context.");
}
}
}
}

128
src/Orchard.Web/Core/Indexing/Lucene/DefaultIndexProvider.cs Normal file
View File

@@ -0,0 +1,128 @@
using System;
using System.IO;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Store;
using Orchard.Environment.Configuration;
using Orchard.FileSystems.AppData;
using Orchard.Indexing;
using Directory = Lucene.Net.Store.Directory;
using Version = Lucene.Net.Util.Version;
using Orchard.Logging;
namespace Orchard.Core.Indexing.Lucene {
/// <summary>
/// Represents the default implementation of an IIndexProvider based on Lucene
/// </summary>
public class DefaultIndexProvider : IIndexProvider {
private readonly IAppDataFolder _appDataFolder;
private readonly ShellSettings _shellSettings;
public static readonly Version LuceneVersion = Version.LUCENE_29;
private readonly Analyzer _analyzer = new StandardAnalyzer(LuceneVersion);
private readonly string _basePath;
public ILogger Logger { get; set; }
public DefaultIndexProvider(IAppDataFolder appDataFolder, ShellSettings shellSettings) {
_appDataFolder = appDataFolder;
_shellSettings = shellSettings;
// TODO: (sebros) Find a common way to get where tenant's specific files should go. "Sites/Tenant" is hard coded in multiple places
_basePath = Path.Combine("Sites", _shellSettings.Name, "Indexes");
Logger = NullLogger.Instance;
// Ensures the directory exists
var directory = new DirectoryInfo(_appDataFolder.MapPath(_basePath));
if(!directory.Exists) {
directory.Create();
}
}
protected virtual Directory GetDirectory(string indexName) {
var directoryInfo = new DirectoryInfo(_appDataFolder.MapPath(Path.Combine(_basePath, indexName)));
return FSDirectory.Open(directoryInfo);
}
private static Document CreateDocument(DefaultIndexDocument indexDocument) {
var doc = new Document();
indexDocument.PrepareForIndexing();
foreach(var field in indexDocument.Fields) {
doc.Add(field);
}
return doc;
}
public bool Exists(string indexName) {
return new DirectoryInfo(_appDataFolder.MapPath(Path.Combine(_basePath, indexName))).Exists;
}
public void CreateIndex(string indexName) {
var writer = new IndexWriter(GetDirectory(indexName), _analyzer, true, IndexWriter.MaxFieldLength.UNLIMITED);
writer.Close();
Logger.Information("Index [{0}] created", indexName);
}
public void DeleteIndex(string indexName) {
new DirectoryInfo(Path.Combine(_appDataFolder.MapPath(Path.Combine(_basePath, indexName))))
.Delete(true);
}
public void Store(string indexName, IIndexDocument indexDocument) {
Store(indexName, (DefaultIndexDocument)indexDocument);
}
public void Store(string indexName, DefaultIndexDocument indexDocument) {
var writer = new IndexWriter(GetDirectory(indexName), _analyzer, false, IndexWriter.MaxFieldLength.UNLIMITED);
try {
var doc = CreateDocument(indexDocument);
writer.AddDocument(doc);
Logger.Debug("Document [{0}] indexed", indexDocument.Id);
}
catch ( Exception ex ) {
Logger.Error(ex, "An unexpected error occured while removing the document [{0}] from the index [{1}].", indexDocument.Id, indexName);
}
finally {
writer.Close();
}
}
public void Delete(string indexName, int id) {
var reader = IndexReader.Open(GetDirectory(indexName), false);
try {
var term = new Term("id", id.ToString());
if ( reader.DeleteDocuments(term) != 0 ) {
Logger.Error("The document [{0}] could not be removed from the index [{1}]", id, indexName);
}
else {
Logger.Debug("Document [{0}] removed from index", id);
}
}
catch ( Exception ex ) {
Logger.Error(ex, "An unexpected error occured while removing the document [{0}] from the index [{1}].", id, indexName);
}
finally {
reader.Close();
}
}
public IIndexDocument New(int documentId) {
return new DefaultIndexDocument(documentId);
}
public ISearchBuilder CreateSearchBuilder(string indexName) {
return new DefaultSearchBuilder(GetDirectory(indexName));
}
public IIndexDocument Get(string indexName, int id) {
throw new NotImplementedException();
}
}
}

194
src/Orchard.Web/Core/Indexing/Lucene/DefaultSearchBuilder.cs Normal file
View File

@@ -0,0 +1,194 @@
using System;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Store;
using Orchard.Logging;
using Lucene.Net.Documents;
using Orchard.Indexing;
namespace Orchard.Core.Indexing.Lucene {
public class DefaultSearchBuilder : ISearchBuilder {
private const int MaxResults = Int16.MaxValue;
private readonly Directory _directory;
private readonly Dictionary<string, Query[]> _fields;
private int _count;
private int _skip;
private readonly Dictionary<string, DateTime> _before;
private readonly Dictionary<string, DateTime> _after;
private string _sort;
private bool _sortDescending;
public ILogger Logger { get; set; }
public DefaultSearchBuilder(Directory directory) {
_directory = directory;
Logger = NullLogger.Instance;
_count = MaxResults;
_skip = 0;
_before = new Dictionary<string, DateTime>();
_after = new Dictionary<string, DateTime>();
_fields = new Dictionary<string, Query[]>();
_sort = String.Empty;
_sortDescending = true;
}
public ISearchBuilder Parse(string query) {
return this;
}
public ISearchBuilder WithField(string field, string value) {
return WithField(field, value, true);
}
public ISearchBuilder WithField(string field, string value, bool wildcardSearch) {
_fields[field] = value.Split(' ')
.Where(k => !String.IsNullOrWhiteSpace(k))
.Select(k => wildcardSearch ? (Query)new PrefixQuery(new Term(field, k)) : new TermQuery(new Term(k)))
.ToArray();
return this;
}
public ISearchBuilder After(string name, DateTime date) {
_after[name] = date;
return this;
}
public ISearchBuilder Before(string name, DateTime date) {
_before[name] = date;
return this;
}
public ISearchBuilder SortBy(string name) {
_sort = name;
return this;
}
public ISearchBuilder Ascending() {
_sortDescending = false;
return this;
}
public ISearchBuilder Slice(int skip, int count) {
if ( skip < 0 ) {
throw new ArgumentException("Skip must be greater or equal to zero");
}
if ( count <= 0 ) {
throw new ArgumentException("Count must be greater than zero");
}
_skip = skip;
_count = count;
return this;
}
private Query CreateQuery() {
var query = new BooleanQuery();
if ( _fields.Keys.Count > 0 ) { // apply specific filters if defined
foreach ( var filters in _fields.Values ) {
foreach(var filter in filters)
query.Add(filter, BooleanClause.Occur.SHOULD);
}
}
// apply date range filter ?
foreach(string name in _before.Keys.Concat(_after.Keys)) {
if ((_before.ContainsKey(name) && _before[name] != DateTime.MaxValue) || (_after.ContainsKey(name) && _after[name] != DateTime.MinValue)) {
var filter = new TermRangeQuery("date",
DateTools.DateToString(_after.ContainsKey(name) ? _after[name] : DateTime.MinValue, DateTools.Resolution.SECOND),
DateTools.DateToString(_before.ContainsKey(name) ? _before[name] : DateTime.MaxValue, DateTools.Resolution.SECOND),
true, true);
query.Add(filter, BooleanClause.Occur.MUST);
}
}
if ( query.Clauses().Count == 0 ) { // get all documents ?
query.Add(new TermRangeQuery("id", "0", "9", true, true), BooleanClause.Occur.SHOULD);
}
Logger.Debug("New search query: {0}", query.ToString());
return query;
}
public IEnumerable<ISearchHit> Search() {
var query = CreateQuery();
var searcher = new IndexSearcher(_directory, true);
try {
var sort = String.IsNullOrEmpty(_sort)
? Sort.RELEVANCE
: new Sort(new SortField(_sort, CultureInfo.InvariantCulture, _sortDescending));
var collector = TopFieldCollector.create(
sort,
_count + _skip,
false,
true,
false,
true);
searcher.Search(query, collector);
var results = new List<DefaultSearchHit>();
foreach ( var scoreDoc in collector.TopDocs().scoreDocs.Skip(_skip) ) {
results.Add(new DefaultSearchHit(searcher.Doc(scoreDoc.doc), scoreDoc.score));
}
Logger.Information("Search results: {0}", results.Count);
return results;
}
finally {
searcher.Close();
}
}
public int Count() {
var query = CreateQuery();
var searcher = new IndexSearcher(_directory, true);
try {
var hits = searcher.Search(query, Int16.MaxValue);
Logger.Information("Search results: {0}", hits.scoreDocs.Length);
var length = hits.scoreDocs.Length;
return Math.Min(length - _skip, _count) ;
}
finally {
searcher.Close();
}
}
public ISearchHit Get(int documentId) {
var query = new TermQuery(new Term("id", documentId.ToString()));
var searcher = new IndexSearcher(_directory, true);
try {
var hits = searcher.Search(query, 1);
Logger.Information("Search results: {0}", hits.scoreDocs.Length);
if ( hits.scoreDocs.Length > 0 ) {
return new DefaultSearchHit(searcher.Doc(hits.scoreDocs[0].doc), hits.scoreDocs[0].score);
}
else {
return null;
}
}
finally {
searcher.Close();
}
}
}
}

40
src/Orchard.Web/Core/Indexing/Lucene/DefaultSearchHit.cs Normal file
View File

@@ -0,0 +1,40 @@
using Lucene.Net.Documents;
using System.Globalization;
using Lucene.Net.Util;
using Orchard.Indexing;
namespace Orchard.Core.Indexing.Lucene {
public class DefaultSearchHit : ISearchHit {
private readonly Document _doc;
private readonly float _score;
public float Score { get { return _score; } }
public DefaultSearchHit(Document document, float score) {
_doc = document;
_score = score;
}
public int Id { get { return int.Parse(GetString("id")); } }
public int GetInt(string name) {
return NumericUtils.PrefixCodedToInt(_doc.GetField(name).StringValue());
}
public float GetFloat(string name) {
return float.Parse(_doc.GetField(name).StringValue(), CultureInfo.InvariantCulture);
}
public bool GetBoolean(string name) {
return bool.Parse(_doc.GetField(name).StringValue());
}
public string GetString(string name) {
return _doc.GetField(name).StringValue();
}
public System.DateTime GetDateTime(string name) {
return DateTools.StringToDate(_doc.GetField(name).StringValue());
}
}
}

8
src/Orchard.Web/Core/Indexing/Models/IndexingSettingsRecord.cs Normal file
View File

@@ -0,0 +1,8 @@
using System;
namespace Orchard.Core.Indexing.Models {
public class IndexingSettingsRecord {
public virtual int Id { get; set; }
public virtual DateTime? LatestIndexingUtc { get; set; }
}
}

36
src/Orchard.Web/Core/Indexing/Models/IndexingTask.cs Normal file
View File

@@ -0,0 +1,36 @@
using System;
using Orchard.ContentManagement;
using Orchard.Tasks.Indexing;
namespace Orchard.Core.Indexing.Models {
public class IndexingTask : IIndexingTask {
private readonly IContentManager _contentManager;
private readonly IndexingTaskRecord _record;
private ContentItem _item;
private bool _itemInitialized;
public IndexingTask(IContentManager contentManager, IndexingTaskRecord record) {
// in spite of appearances, this is actually a created class, not IoC,
// but dependencies are passed in for lazy initialization purposes
_contentManager = contentManager;
_record = record;
}
public DateTime? CreatedUtc {
get { return _record.CreatedUtc; }
}
public ContentItem ContentItem {
get {
if (!_itemInitialized) {
if (_record.ContentItemRecord != null) {
_item = _contentManager.Get(
_record.ContentItemRecord.Id, VersionOptions.Published);
}
_itemInitialized = true;
}
return _item;
}
}
}
}

10
src/Orchard.Web/Core/Indexing/Models/IndexingTaskRecord.cs Normal file
View File

@@ -0,0 +1,10 @@
using System;
using Orchard.ContentManagement.Records;
namespace Orchard.Core.Indexing.Models {
public class IndexingTaskRecord {
public virtual int Id { get; set; }
public virtual DateTime? CreatedUtc { get; set; }
public virtual ContentItemRecord ContentItemRecord { get; set; }
}
}

10
src/Orchard.Web/Core/Indexing/Module.txt Normal file
View File

@@ -0,0 +1,10 @@
name: Indexing
antiforgery: enabled
author: The Orchard Team
website: http://orchardproject.net
version: 0.1
orchardversion: 0.1.2010.0312
features:
Indexing:
Description: Indexing services based on Lucene.
Category: Core

30
src/Orchard.Web/Core/Indexing/Services/CreateIndexingTaskHandler.cs Normal file
View File

@@ -0,0 +1,30 @@
using Orchard.ContentManagement.Handlers;
using Orchard.ContentManagement;
using Orchard.Core.Common.Models;
using Orchard.Tasks.Indexing;
namespace Orchard.Core.Indexing.Services {
/// <summary>
/// Intercepts the ContentHandler events to create indexing tasks when a content item
/// is published, and to delete them when the content item is unpublished.
/// </summary>
public class CreateIndexingTaskHandler : ContentHandler {
private readonly IIndexingTaskManager _indexingTaskManager;
public CreateIndexingTaskHandler(IIndexingTaskManager indexingTaskManager) {
_indexingTaskManager = indexingTaskManager;
OnPublishing<ContentPart<CommonRecord>>(CreateIndexingTask);
OnRemoved<ContentPart<CommonRecord>>(RemoveIndexingTask);
}
void CreateIndexingTask(PublishContentContext context, ContentPart<CommonRecord> part) {
_indexingTaskManager.CreateTask(context.ContentItem);
}
void RemoveIndexingTask(RemoveContentContext context, ContentPart<CommonRecord> part) {
_indexingTaskManager.DeleteTasks(context.ContentItem);
}
}
}

108
src/Orchard.Web/Core/Indexing/Services/IndexingTaskExecutor.cs Normal file
View File

@@ -0,0 +1,108 @@
using System;
using System.Collections.Generic;
using System.Linq;
using JetBrains.Annotations;
using Orchard.ContentManagement;
using Orchard.ContentManagement.Handlers;
using Orchard.Data;
using Orchard.Indexing;
using Orchard.Logging;
using Orchard.Services;
using Orchard.Tasks;
using Orchard.Core.Indexing.Models;
namespace Orchard.Core.Indexing.Services {
/// <summary>
/// Contains the logic which is regularly executed to retrieve index information from multiple content handlers.
/// </summary>
[UsedImplicitly]
public class IndexingTaskExecutor : IBackgroundTask {
private readonly IClock _clock;
private readonly IRepository<IndexingTaskRecord> _repository;
private readonly IRepository<IndexingSettingsRecord> _settings;
private readonly IEnumerable<IContentHandler> _handlers;
private IIndexProvider _indexProvider;
private IIndexManager _indexManager;
private readonly IContentManager _contentManager;
private const string SearchIndexName = "search";
public IndexingTaskExecutor(
IClock clock,
IRepository<IndexingTaskRecord> repository,
IRepository<IndexingSettingsRecord> settings,
IEnumerable<IContentHandler> handlers,
IIndexManager indexManager,
IContentManager contentManager) {
_clock = clock;
_repository = repository;
_settings = settings;
_indexManager = indexManager;
_handlers = handlers;
_contentManager = contentManager;
Logger = NullLogger.Instance;
}
public ILogger Logger { get; set; }
public void Sweep() {
if(!_indexManager.HasIndexProvider()) {
return;
}
_indexProvider = _indexManager.GetSearchIndexProvider();
// retrieve last processed index time
var settingsRecord = _settings.Table.FirstOrDefault();
if (settingsRecord == null) {
_settings.Create(settingsRecord = new IndexingSettingsRecord { LatestIndexingUtc = new DateTime(1980, 1, 1)});
}
var lastIndexing = settingsRecord.LatestIndexingUtc;
settingsRecord.LatestIndexingUtc = _clock.UtcNow;
// retrieved not yet processed tasks
var taskRecords = _repository.Fetch(x => x.CreatedUtc >= lastIndexing)
.ToArray();
if (taskRecords.Length == 0)
return;
Logger.Information("Processing {0} indexing tasks", taskRecords.Length);
if(!_indexProvider.Exists(SearchIndexName)) {
_indexProvider.CreateIndex(SearchIndexName);
}
foreach (var taskRecord in taskRecords) {
try {
var task = new IndexingTask(_contentManager, taskRecord);
var context = new IndexContentContext {
ContentItem = task.ContentItem,
IndexDocument = _indexProvider.New(task.ContentItem.Id)
};
// dispatch to handlers to retrieve index information
foreach (var handler in _handlers) {
handler.Indexing(context);
}
_indexProvider.Store(SearchIndexName, context.IndexDocument);
foreach ( var handler in _handlers ) {
handler.Indexed(context);
}
}
catch (Exception ex) {
Logger.Warning(ex, "Unable to process indexing task #{0}", taskRecord.Id);
}
}
_settings.Update(settingsRecord);
}
}
}

101
src/Orchard.Web/Core/Indexing/Services/IndexingTaskManager.cs Normal file
View File

@@ -0,0 +1,101 @@
using System;
using System.Collections.Generic;
using System.Linq;
using JetBrains.Annotations;
using Orchard.ContentManagement;
using Orchard.Data;
using Orchard.Logging;
using Orchard.Tasks.Scheduling;
using Orchard.Utility.Extensions;
using Orchard.Tasks.Indexing;
using Orchard.Core.Indexing.Models;
using Orchard.Services;
namespace Orchard.Core.Indexing.Services {
[UsedImplicitly]
public class IndexingTaskManager : IIndexingTaskManager {
private readonly IContentManager _contentManager;
private readonly IRepository<IndexingTaskRecord> _repository;
private readonly IRepository<IndexingSettingsRecord> _settings;
private readonly IClock _clock;
public IndexingTaskManager(
IContentManager contentManager,
IRepository<IndexingTaskRecord> repository,
IRepository<IndexingSettingsRecord> settings,
IClock clock) {
_clock = clock;
_repository = repository;
_contentManager = contentManager;
Logger = NullLogger.Instance;
}
public ILogger Logger { get; set; }
public void CreateTask(ContentItem contentItem) {
if (contentItem == null) {
throw new ArgumentNullException("contentItem");
}
// remove previous tasks for the same content item
var tasks = _repository
.Fetch(x => x.Id == contentItem.Id )
.ToArray();
foreach (var task in tasks) {
_repository.Delete(task);
}
var taskRecord = new IndexingTaskRecord {
CreatedUtc = _clock.UtcNow,
ContentItemRecord = contentItem.Record
};
_repository.Create(taskRecord);
Logger.Information("Indexing task created for [{0}:{1}]", contentItem.ContentType, contentItem.Id);
}
public IEnumerable<IIndexingTask> GetTasks(DateTime? createdAfter) {
return _repository
.Fetch(x => x.CreatedUtc > createdAfter)
.Select(x => new IndexingTask(_contentManager, x))
.Cast<IIndexingTask>()
.ToReadOnlyCollection();
}
public void DeleteTasks(DateTime? createdBefore) {
Logger.Debug("Deleting Indexing tasks created before {0}", createdBefore);
var tasks = _repository
.Fetch(x => x.CreatedUtc <= createdBefore);
foreach (var task in tasks) {
_repository.Delete(task);
}
}
public void DeleteTasks(ContentItem contentItem) {
Logger.Debug("Deleting Indexing tasks for ContentItem [{0}:{1}]", contentItem.ContentType, contentItem.Id);
var tasks = _repository
.Fetch(x => x.Id == contentItem.Id);
foreach (var task in tasks) {
_repository.Delete(task);
}
}
public void RebuildIndex() {
var settingsRecord = _settings.Table.FirstOrDefault();
if (settingsRecord == null) {
_settings.Create(settingsRecord = new IndexingSettingsRecord() );
}
settingsRecord.LatestIndexingUtc = new DateTime(1980, 1, 1);
_settings.Update(settingsRecord);
}
}
}

15
src/Orchard.Web/Core/Orchard.Core.csproj
View File

@@ -39,6 +39,10 @@
<CodeAnalysisRuleSet>AllRules.ruleset</CodeAnalysisRuleSet> <CodeAnalysisRuleSet>AllRules.ruleset</CodeAnalysisRuleSet>
</PropertyGroup> </PropertyGroup>
<ItemGroup> <ItemGroup>
<Reference Include="Lucene.Net, Version=2.9.2.2, Culture=neutral, processorArchitecture=MSIL">
<SpecificVersion>False</SpecificVersion>
<HintPath>..\..\..\lib\lucene.net\Lucene.Net.dll</HintPath>
</Reference>
<Reference Include="System" /> <Reference Include="System" />
<Reference Include="System.ComponentModel.DataAnnotations"> <Reference Include="System.ComponentModel.DataAnnotations">
<RequiredTargetFramework>3.5</RequiredTargetFramework> <RequiredTargetFramework>3.5</RequiredTargetFramework>
@@ -105,6 +109,16 @@
<Compile Include="Feeds\Rss\RssResult.cs" /> <Compile Include="Feeds\Rss\RssResult.cs" />
<Compile Include="HomePage\Controllers\HomeController.cs" /> <Compile Include="HomePage\Controllers\HomeController.cs" />
<Compile Include="HomePage\Routes.cs" /> <Compile Include="HomePage\Routes.cs" />
<Compile Include="Indexing\Lucene\DefaultIndexDocument.cs" />
<Compile Include="Indexing\Lucene\DefaultIndexProvider.cs" />
<Compile Include="Indexing\Lucene\DefaultSearchBuilder.cs" />
<Compile Include="Indexing\Lucene\DefaultSearchHit.cs" />
<Compile Include="Indexing\Models\IndexingSettingsRecord.cs" />
<Compile Include="Indexing\Models\IndexingTask.cs" />
<Compile Include="Indexing\Models\IndexingTaskRecord.cs" />
<Compile Include="Indexing\Services\CreateIndexingTaskHandler.cs" />
<Compile Include="Indexing\Services\IndexingTaskExecutor.cs" />
<Compile Include="Indexing\Services\IndexingTaskManager.cs" />
<Compile Include="Navigation\AdminMenu.cs" /> <Compile Include="Navigation\AdminMenu.cs" />
<Compile Include="Navigation\Controllers\AdminController.cs" /> <Compile Include="Navigation\Controllers\AdminController.cs" />
<Compile Include="Navigation\Models\MenuItem.cs" /> <Compile Include="Navigation\Models\MenuItem.cs" />
@@ -159,6 +173,7 @@
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>
<Content Include="Common\Module.txt" /> <Content Include="Common\Module.txt" />
<Content Include="Indexing\Module.txt" />
<Content Include="Settings\Module.txt" /> <Content Include="Settings\Module.txt" />
<Content Include="Settings\Views\Admin\Index.ascx" /> <Content Include="Settings\Views\Admin\Index.ascx" />
<Content Include="Web.config" /> <Content Include="Web.config" />

1
src/Orchard.Web/Modules/Orchard.Setup/Services/SetupService.cs
View File

@@ -60,6 +60,7 @@ namespace Orchard.Setup.Services {
"HomePage", "HomePage",
"Navigation", "Navigation",
"Scheduling", "Scheduling",
"Indexing",
"Settings", "Settings",
"XmlRpc", "XmlRpc",
"Orchard.Users", "Orchard.Users",

2
src/Orchard/ContentManagement/Drivers/ContentItemDriverHandler.cs
View File

@@ -33,6 +33,8 @@ namespace Orchard.ContentManagement.Drivers {
void IContentHandler.Published(PublishContentContext context) { } void IContentHandler.Published(PublishContentContext context) { }
void IContentHandler.Removing(RemoveContentContext context) { } void IContentHandler.Removing(RemoveContentContext context) { }
void IContentHandler.Removed(RemoveContentContext context) { } void IContentHandler.Removed(RemoveContentContext context) { }
void IContentHandler.Indexing(IndexContentContext context) { }
void IContentHandler.Indexed(IndexContentContext context) { }
void IContentHandler.GetContentItemMetadata(GetContentItemMetadataContext context) { void IContentHandler.GetContentItemMetadata(GetContentItemMetadataContext context) {

2
src/Orchard/ContentManagement/Drivers/ContentPartDriverHandler.cs
View File

@@ -32,6 +32,8 @@ namespace Orchard.ContentManagement.Drivers {
void IContentHandler.Published(PublishContentContext context) { } void IContentHandler.Published(PublishContentContext context) { }
void IContentHandler.Removing(RemoveContentContext context) { } void IContentHandler.Removing(RemoveContentContext context) { }
void IContentHandler.Removed(RemoveContentContext context) { } void IContentHandler.Removed(RemoveContentContext context) { }
void IContentHandler.Indexing(IndexContentContext context) { }
void IContentHandler.Indexed(IndexContentContext context) { }
void IContentHandler.GetContentItemMetadata(GetContentItemMetadataContext context) { } void IContentHandler.GetContentItemMetadata(GetContentItemMetadataContext context) { }

33
src/Orchard/ContentManagement/Handlers/ContentHandler.cs
View File

@@ -57,6 +57,14 @@ namespace Orchard.ContentManagement.Handlers {
Filters.Add(new InlineStorageFilter<TPart> { OnRemoved = handler }); Filters.Add(new InlineStorageFilter<TPart> { OnRemoved = handler });
} }
protected void OnIndexing<TPart>(Action<IndexContentContext, TPart> handler) where TPart : class, IContent {
Filters.Add(new InlineStorageFilter<TPart> { OnIndexing = handler });
}
protected void OnIndexed<TPart>(Action<IndexContentContext, TPart> handler) where TPart : class, IContent {
Filters.Add(new InlineStorageFilter<TPart> { OnIndexed = handler });
}
protected void OnGetContentItemMetadata<TPart>(Action<GetContentItemMetadataContext, TPart> handler) where TPart : class, IContent { protected void OnGetContentItemMetadata<TPart>(Action<GetContentItemMetadataContext, TPart> handler) where TPart : class, IContent {
Filters.Add(new InlineTemplateFilter<TPart> { OnGetItemMetadata = handler }); Filters.Add(new InlineTemplateFilter<TPart> { OnGetItemMetadata = handler });
} }
@@ -84,6 +92,8 @@ namespace Orchard.ContentManagement.Handlers {
public Action<PublishContentContext, TPart> OnPublished { get; set; } public Action<PublishContentContext, TPart> OnPublished { get; set; }
public Action<RemoveContentContext, TPart> OnRemoving { get; set; } public Action<RemoveContentContext, TPart> OnRemoving { get; set; }
public Action<RemoveContentContext, TPart> OnRemoved { get; set; } public Action<RemoveContentContext, TPart> OnRemoved { get; set; }
public Action<IndexContentContext, TPart> OnIndexing { get; set; }
public Action<IndexContentContext, TPart> OnIndexed { get; set; }
protected override void Activated(ActivatedContentContext context, TPart instance) { protected override void Activated(ActivatedContentContext context, TPart instance) {
if (OnActivated != null) OnActivated(context, instance); if (OnActivated != null) OnActivated(context, instance);
} }
@@ -117,6 +127,15 @@ namespace Orchard.ContentManagement.Handlers {
protected override void Removed(RemoveContentContext context, TPart instance) { protected override void Removed(RemoveContentContext context, TPart instance) {
if (OnRemoved != null) OnRemoved(context, instance); if (OnRemoved != null) OnRemoved(context, instance);
} }
protected override void Indexing(IndexContentContext context, TPart instance) {
if ( OnIndexing != null )
OnIndexing(context, instance);
}
protected override void Indexed(IndexContentContext context, TPart instance) {
if ( OnIndexed != null )
OnIndexed(context, instance);
}
} }
class InlineTemplateFilter<TPart> : TemplateFilterBase<TPart> where TPart : class, IContent { class InlineTemplateFilter<TPart> : TemplateFilterBase<TPart> where TPart : class, IContent {
@@ -214,6 +233,17 @@ namespace Orchard.ContentManagement.Handlers {
Removed(context); Removed(context);
} }
void IContentHandler.Indexing(IndexContentContext context) {
foreach ( var filter in Filters.OfType<IContentStorageFilter>() )
filter.Indexing(context);
Indexing(context);
}
void IContentHandler.Indexed(IndexContentContext context) {
foreach ( var filter in Filters.OfType<IContentStorageFilter>() )
filter.Indexed(context);
Indexing(context);
}
void IContentHandler.GetContentItemMetadata(GetContentItemMetadataContext context) { void IContentHandler.GetContentItemMetadata(GetContentItemMetadataContext context) {
foreach (var filter in Filters.OfType<IContentTemplateFilter>()) foreach (var filter in Filters.OfType<IContentTemplateFilter>())
@@ -254,6 +284,9 @@ namespace Orchard.ContentManagement.Handlers {
protected virtual void Removing(RemoveContentContext context) { } protected virtual void Removing(RemoveContentContext context) { }
protected virtual void Removed(RemoveContentContext context) { } protected virtual void Removed(RemoveContentContext context) { }
protected virtual void Indexing(IndexContentContext context) { }
protected virtual void Indexed(IndexContentContext context) { }
protected virtual void GetItemMetadata(GetContentItemMetadataContext context) { } protected virtual void GetItemMetadata(GetContentItemMetadataContext context) { }
protected virtual void BuildDisplayModel(BuildDisplayModelContext context) { } protected virtual void BuildDisplayModel(BuildDisplayModelContext context) { }
protected virtual void BuildEditorModel(BuildEditorModelContext context) { } protected virtual void BuildEditorModel(BuildEditorModelContext context) { }

2
src/Orchard/ContentManagement/Handlers/IContentHandler.cs
View File

@@ -17,6 +17,8 @@ namespace Orchard.ContentManagement.Handlers {
void Published(PublishContentContext context); void Published(PublishContentContext context);
void Removing(RemoveContentContext context); void Removing(RemoveContentContext context);
void Removed(RemoveContentContext context); void Removed(RemoveContentContext context);
void Indexing(IndexContentContext context);
void Indexed(IndexContentContext context);
void GetContentItemMetadata(GetContentItemMetadataContext context); void GetContentItemMetadata(GetContentItemMetadataContext context);
void BuildDisplayModel(BuildDisplayModelContext context); void BuildDisplayModel(BuildDisplayModelContext context);

2
src/Orchard/ContentManagement/Handlers/IContentStorageFilter.cs
View File

@@ -11,5 +11,7 @@ namespace Orchard.ContentManagement.Handlers {
void Published(PublishContentContext context); void Published(PublishContentContext context);
void Removing(RemoveContentContext context); void Removing(RemoveContentContext context);
void Removed(RemoveContentContext context); void Removed(RemoveContentContext context);
void Indexing(IndexContentContext context);
void Indexed(IndexContentContext context);
} }
} }

8
src/Orchard/ContentManagement/Handlers/IndexContentContext.cs Normal file
View File

@@ -0,0 +1,8 @@
using Orchard.Indexing;
namespace Orchard.ContentManagement.Handlers {
public class IndexContentContext {
public ContentItem ContentItem { get; set; }
public IIndexDocument IndexDocument { get; set; }
}
}

13
src/Orchard/ContentManagement/Handlers/StorageFilterBase.cs
View File

@@ -12,6 +12,8 @@ namespace Orchard.ContentManagement.Handlers {
protected virtual void Published(PublishContentContext context, TPart instance) { } protected virtual void Published(PublishContentContext context, TPart instance) { }
protected virtual void Removing(RemoveContentContext context, TPart instance) { } protected virtual void Removing(RemoveContentContext context, TPart instance) { }
protected virtual void Removed(RemoveContentContext context, TPart instance) { } protected virtual void Removed(RemoveContentContext context, TPart instance) { }
protected virtual void Indexing(IndexContentContext context, TPart instance) { }
protected virtual void Indexed(IndexContentContext context, TPart instance) { }
void IContentStorageFilter.Activated(ActivatedContentContext context) { void IContentStorageFilter.Activated(ActivatedContentContext context) {
@@ -68,5 +70,16 @@ namespace Orchard.ContentManagement.Handlers {
if (context.ContentItem.Is<TPart>()) if (context.ContentItem.Is<TPart>())
Removed(context, context.ContentItem.As<TPart>()); Removed(context, context.ContentItem.As<TPart>());
} }
void IContentStorageFilter.Indexing(IndexContentContext context) {
if ( context.ContentItem.Is<TPart>() )
Indexing(context, context.ContentItem.As<TPart>());
}
void IContentStorageFilter.Indexed(IndexContentContext context) {
if ( context.ContentItem.Is<TPart>() )
Indexed(context, context.ContentItem.As<TPart>());
}
} }
} }

1
src/Orchard/Environment/OrchardStarter.cs
View File

@@ -4,7 +4,6 @@ using System.IO;
using System.Web.Hosting; using System.Web.Hosting;
using Autofac; using Autofac;
using Autofac.Configuration; using Autofac.Configuration;
using Autofac.Integration.Web;
using Orchard.Caching; using Orchard.Caching;
using Orchard.Environment.AutofacUtil; using Orchard.Environment.AutofacUtil;
using Orchard.Environment.Configuration; using Orchard.Environment.Configuration;

25
src/Orchard/Indexing/DefaultIndexManager.cs Normal file
View File

@@ -0,0 +1,25 @@
using System.Collections.Generic;
using System.Linq;
namespace Orchard.Indexing {
public class DefaultIndexManager : IIndexManager {
private readonly IEnumerable<IIndexProvider> _indexProviders;
public DefaultIndexManager(IEnumerable<IIndexProvider> indexProviders) {
_indexProviders = indexProviders;
}
#region IIndexManager Members
public bool HasIndexProvider() {
return _indexProviders.AsQueryable().Count() > 0;
}
public IIndexProvider GetSearchIndexProvider() {
return _indexProviders.AsQueryable().FirstOrDefault();
}
#endregion
}
}

27
src/Orchard/Indexing/IIndexDocument.cs Normal file
View File

@@ -0,0 +1,27 @@
using System;
using System.Collections.Generic;
namespace Orchard.Indexing {
public interface IIndexDocument {
IIndexDocument SetContentItemId(int documentId);
IIndexDocument Add(string name, string value);
IIndexDocument Add(string name, DateTime value);
IIndexDocument Add(string name, int value);
IIndexDocument Add(string name, bool value);
IIndexDocument Add(string name, float value);
/// <summary>
/// Whether to store the original value to the index
/// </summary>
IIndexDocument Store(bool store);
/// <summary>
/// Whether the content should be tokenized or not. If not, value will be taken as a whole
/// </summary>
IIndexDocument Analyze(bool analyze);
}
}

7
src/Orchard/Indexing/IIndexManager.cs Normal file
View File

@@ -0,0 +1,7 @@
namespace Orchard.Indexing {
public interface IIndexManager : IDependency {
bool HasIndexProvider();
IIndexProvider GetSearchIndexProvider();
}
}

45
src/Orchard/Indexing/IIndexProvider.cs Normal file
View File

@@ -0,0 +1,45 @@
namespace Orchard.Indexing {
public interface IIndexProvider : IDependency {
/// <summary>
/// Creates a new index
/// </summary>
void CreateIndex(string name);
/// <summary>
/// Checks whether an index is already existing or not
/// </summary>
bool Exists(string name);
/// <summary>
/// Deletes an existing index
/// </summary>
void DeleteIndex(string name);
/// <summary>
/// Loads an existing document
/// </summary>
IIndexDocument Get(string indexName, int documentId);
/// <summary>
/// Creates an empty document
/// </summary>
/// <returns></returns>
IIndexDocument New(int documentId);
/// <summary>
/// Adds a new document to the index
/// </summary>
void Store(string indexName, IIndexDocument indexDocument);
/// <summary>
/// Removes an existing document from the index
/// </summary>
void Delete(string indexName, int id);
/// <summary>
/// Creates a search builder for this provider
/// </summary>
/// <returns>A search builder instance</returns>
ISearchBuilder CreateSearchBuilder(string indexName);
}
}

24
src/Orchard/Indexing/ISearchBuilder.cs Normal file
View File

@@ -0,0 +1,24 @@
using System;
using System.Collections.Generic;
namespace Orchard.Indexing {
public interface ISearchBuilder {
ISearchBuilder Parse(string query);
ISearchBuilder WithField(string field, string value);
ISearchBuilder WithField(string field, string value, bool wildcardSearch);
ISearchBuilder After(string name, DateTime date);
ISearchBuilder Before(string name, DateTime date);
ISearchBuilder SortBy(string name);
ISearchBuilder Ascending();
ISearchBuilder Slice(int skip, int count);
IEnumerable<ISearchHit> Search();
ISearchHit Get(int documentId);
int Count();
}
}

13
src/Orchard/Indexing/ISearchHit.cs Normal file
View File

@@ -0,0 +1,13 @@
using System;
namespace Orchard.Indexing {
public interface ISearchHit {
int Id { get; }
float Score { get; }
int GetInt(string name);
float GetFloat(string name);
bool GetBoolean(string name);
string GetString(string name);
DateTime GetDateTime(string name);
}
}

9
src/Orchard/Orchard.Framework.csproj
View File

@@ -145,6 +145,8 @@
<Compile Include="IDependency.cs" /> <Compile Include="IDependency.cs" />
<Compile Include="Localization\Services\DefaultCultureManager.cs" /> <Compile Include="Localization\Services\DefaultCultureManager.cs" />
<Compile Include="Localization\Services\DefaultResourceManager.cs" /> <Compile Include="Localization\Services\DefaultResourceManager.cs" />
<Compile Include="Indexing\DefaultIndexManager.cs" />
<Compile Include="Indexing\IIndexManager.cs" />
<Compile Include="Localization\Services\ICultureManager.cs" /> <Compile Include="Localization\Services\ICultureManager.cs" />
<Compile Include="Localization\Services\ICultureSelector.cs" /> <Compile Include="Localization\Services\ICultureSelector.cs" />
<Compile Include="Localization\Services\IResourceManager.cs" /> <Compile Include="Localization\Services\IResourceManager.cs" />
@@ -163,6 +165,13 @@
<Compile Include="Mvc\OrchardControllerFactory.cs" /> <Compile Include="Mvc\OrchardControllerFactory.cs" />
<Compile Include="Environment\IOrchardHost.cs" /> <Compile Include="Environment\IOrchardHost.cs" />
<Compile Include="Properties\AssemblyInfo.cs" /> <Compile Include="Properties\AssemblyInfo.cs" />
<Compile Include="Indexing\IIndexDocument.cs" />
<Compile Include="Indexing\IIndexProvider.cs" />
<Compile Include="Indexing\ISearchBuilder.cs" />
<Compile Include="Indexing\ISearchHit.cs" />
<Compile Include="Tasks\Indexing\IIndexingTask.cs" />
<Compile Include="Tasks\Indexing\IIndexingTaskManager.cs" />
<Compile Include="ContentManagement\Handlers\IndexContentContext.cs" />
<Compile Include="Validation\Argument.cs" /> <Compile Include="Validation\Argument.cs" />
</ItemGroup> </ItemGroup>
<ItemGroup> <ItemGroup>

9
src/Orchard/Tasks/Indexing/IIndexingTask.cs Normal file
View File

@@ -0,0 +1,9 @@
using System;
using Orchard.ContentManagement;
namespace Orchard.Tasks.Indexing {
public interface IIndexingTask {
ContentItem ContentItem { get; }
DateTime? CreatedUtc { get; }
}
}

12
src/Orchard/Tasks/Indexing/IIndexingTaskManager.cs Normal file
View File

@@ -0,0 +1,12 @@
using System;
using System.Collections.Generic;
using Orchard.ContentManagement;
namespace Orchard.Tasks.Indexing {
public interface IIndexingTaskManager : IDependency {
void CreateTask(ContentItem contentItem);
IEnumerable<IIndexingTask> GetTasks(DateTime? createdAfter);
void DeleteTasks(DateTime? createdBefore);
void DeleteTasks(ContentItem contentItem);
}
}

2
src/Orchard/Tasks/SweepGenerator.cs
View File

@@ -7,7 +7,7 @@ using Orchard.Logging;
namespace Orchard.Tasks { namespace Orchard.Tasks {
public class SweepGenerator : IOrchardShellEvents { public class SweepGenerator : IOrchardShellEvents {
private readonly IContainer _container; private readonly IContainer _container;
private Timer _timer; private readonly Timer _timer;
public SweepGenerator(IContainer container) { public SweepGenerator(IContainer container) {
_container = container; _container = container;