-
Notifications
You must be signed in to change notification settings - Fork 675
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SOLR-15242: Consolidate README.md with solr/README.md #610
Changes from 11 commits
da173d3
9cf87bd
b67ad39
2a2a32a
0911f87
dfa3089
bd29528
6877151
e7b5af3
8426d43
cf224ad
cf6e6e8
3024f20
160b000
b43af2e
f299706
5825539
36e2fdc
28bdc78
a31f301
f9d6535
950b5a3
657c5c2
7bc921f
27d85c4
a465efe
4c35ef4
e90cec1
e79a432
67c3f0a
ee66008
7c93dae
c0120a2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,307 @@ | ||
<!-- | ||
Licensed to the Apache Software Foundation (ASF) under one or more | ||
contributor license agreements. See the NOTICE file distributed with | ||
this work for additional information regarding copyright ownership. | ||
The ASF licenses this file to You under the Apache License, Version 2.0 | ||
the "License"); you may not use this file except in compliance with | ||
the License. You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
--> | ||
|
||
# Welcome to the Apache Solr project! | ||
----------------------------------- | ||
|
||
Solr is the popular, blazing fast open source enterprise search platform | ||
written in Java and using [Apache Lucene](https://lucene.apache.org/). | ||
|
||
For a complete description of the Solr project, team composition, source | ||
code repositories, and other details, please see the Solr web site at | ||
https://solr.apache.org/ | ||
|
||
## Online Documentation | ||
|
||
This README file only contains basic instructions. For comprehensive documentation, | ||
visit the [Solr Reference Guide](https://solr.apache.org/guide/). | ||
|
||
## Getting Started | ||
FIXME maybe put the tutorial that Noble or Ishan wrote??? Then at the end put the examples? | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [Q] Alternatively - do we really want a full blown tutorial in our README? At best, it duplicates documentation we have elsewhere. At worst it makes it harder to find the information the reader actually came for. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree that we shouldn't have a full-blown tutorial here. We should have links to our actual tutorials, and enhance them if need be. If we do keep a tutorial here, let's put other stuff before it, like the docker and kubernetes sections. These are smaller and are really just advertising other documentation/sites that people should be checking out. |
||
|
||
### Starting Solr | ||
|
||
Start a Solr node in cluster mode (SolrCloud mode) | ||
|
||
``` | ||
bin/solr -c | ||
``` | ||
|
||
To start another Solr node and have it join the cluster alongside the first node, | ||
|
||
``` | ||
bin/solr -c -z localhost:9983 -p 8984 | ||
``` | ||
|
||
An instance of the cluster coordination service, i.e. Zookeeper, was started on port 9983 when the first node was started. To start Zookeeper separately, please refer to XXXX. | ||
|
||
### Creating a collection | ||
|
||
Like a database system holds data in tables, Solr holds data in collections. A collection can be created as follows: | ||
|
||
``` | ||
curl --request POST \ | ||
--url http://localhost:8983/api/collections \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [+1] Thanks for the effort to use the v2 APIs! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [-0] If we are trying to fixup the v2 APIs before marketing them, lets unfortunately continue to use the v1 APIs until they are ready. (If we plan on keeping the tutorial section) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I guess that makes sense, unfortunately. I'd love to get these APIs mostly locked in as soon as we can though, so that we can start pushing them. On that note: anyone reading this should go review the proposed changes to the v2 API here 😛 |
||
--header 'Content-Type: application/json' \ | ||
--data '{ | ||
"create": { | ||
"name": "techproducts", | ||
"numShards": 1, | ||
"replicationFactor": 1 | ||
} | ||
}' | ||
``` | ||
|
||
### Defining a schema | ||
|
||
Let us define some of the fields that our documents will contain. | ||
|
||
``` | ||
curl --request POST \ | ||
--url http://localhost:8983/api/collections/techproducts/schema \ | ||
--header 'Content-Type: application/json' \ | ||
--data '{ | ||
"add-field": [ | ||
{"name": "name", "type": "text_general", "multiValued": false}, | ||
{"name": "cat", "type": "string", "multiValued": true}, | ||
{"name": "manu", "type": "string"}, | ||
{"name": "features", "type": "text_general", "multiValued": true}, | ||
{"name": "weight", "type": "pfloat"}, | ||
{"name": "price", "type": "pfloat"}, | ||
{"name": "popularity", "type": "pint"}, | ||
{"name": "inStock", "type": "boolean", "stored": true}, | ||
{"name": "store", "type": "location"} | ||
] | ||
}' | ||
``` | ||
|
||
### Indexing documents | ||
|
||
A single document can be indexed as: | ||
|
||
``` | ||
curl --request POST \ | ||
--url 'http://localhost:8983/api/collections/techproducts/update' \ | ||
--header 'Content-Type: application/json' \ | ||
--data ' { | ||
"id" : "978-0641723445", | ||
"cat" : ["book","hardcover"], | ||
"name" : "The Lightning Thief", | ||
"author" : "Rick Riordan", | ||
"series_t" : "Percy Jackson and the Olympians", | ||
"sequence_i" : 1, | ||
"genre_s" : "fantasy", | ||
"inStock" : true, | ||
"price" : 12.50, | ||
"pages_i" : 384 | ||
}' | ||
``` | ||
|
||
Multiple documents can be indexed in the same request: | ||
``` | ||
curl --request POST \ | ||
--url 'http://localhost:8983/api/collections/techproducts/update' \ | ||
--header 'Content-Type: application/json' \ | ||
--data ' [ | ||
{ | ||
"id" : "978-0641723445", | ||
"cat" : ["book","hardcover"], | ||
"name" : "The Lightning Thief", | ||
"author" : "Rick Riordan", | ||
"series_t" : "Percy Jackson and the Olympians", | ||
"sequence_i" : 1, | ||
"genre_s" : "fantasy", | ||
"inStock" : true, | ||
"price" : 12.50, | ||
"pages_i" : 384 | ||
} | ||
, | ||
{ | ||
"id" : "978-1423103349", | ||
"cat" : ["book","paperback"], | ||
"name" : "The Sea of Monsters", | ||
"author" : "Rick Riordan", | ||
"series_t" : "Percy Jackson and the Olympians", | ||
"sequence_i" : 2, | ||
"genre_s" : "fantasy", | ||
"inStock" : true, | ||
"price" : 6.49, | ||
"pages_i" : 304 | ||
} | ||
]' | ||
``` | ||
|
||
A file containing the documents can be indexed as follows: | ||
``` | ||
curl -H "Content-Type: application/json" \ | ||
-X POST \ | ||
-d @example/products.json \ | ||
--url 'http://localhost:8983/api/collections/techproducts/update?commit=true' | ||
``` | ||
|
||
### Commit | ||
After documents are indexed into a collection, they are not immediately available for searching. In order to have them searchable, a commit operation (also called `refresh` in other search engines like OpenSearch etc.) is needed. Commits can be scheduled at periodic intervals using auto-commits as follows. | ||
|
||
``` | ||
curl -X POST -H 'Content-type: application/json' -d '{"set-property":{"updateHandler.autoCommit.maxTime":15000}}' http://localhost:8983/api/collections/techproducts/config | ||
``` | ||
|
||
### Basic search queries | ||
FIXME | ||
|
||
### Solr Examples | ||
|
||
Solr includes a few examples to help you get started. To run a specific example, enter: | ||
|
||
``` | ||
bin/solr -e <EXAMPLE> where <EXAMPLE> is one of: | ||
cloud: SolrCloud example | ||
techproducts: Comprehensive example illustrating many of Solr's core capabilities | ||
schemaless: Schema-less example (schema is inferred from data during indexing) | ||
films: Example of starting with _default configset and adding explicit fields dynamically | ||
``` | ||
|
||
For instance, if you want to run the techproducts example, enter: | ||
|
||
``` | ||
bin/solr -e techproducts | ||
``` | ||
|
||
### Running Solr in Docker | ||
|
||
You can run Solr in Docker via the [official image](https://hub.docker.com/_/solr). | ||
|
||
To run Solr in a container and expose the Solr port, run: | ||
|
||
`docker run -p 8983:8983 solr` | ||
|
||
In order to start Solr in cloud mode, run the following. | ||
|
||
`docker run -p 8983:8983 solr solr-fg -c` | ||
|
||
For documentation on using the official docker builds, please refer to the [DockerHub page](https://hub.docker.com/_/solr). | ||
Up to date documentation for running locally built images of this branch can be found in the [local reference guide](solr/solr-ref-guide/src/running-solr-in-docker.adoc). | ||
|
||
There is also a gradle task for building custom Solr images from your local checkout. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Perhaps add a link to Hmm, looking a few lines down, I now see that There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm going to take this as "it's okay" for now then on this comment? Quite agree on your frustration! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry for that placement, there was discussion at the time, but it is long forgotten. I'd be happy to move it to wherever it will make the most sense 🙂 |
||
These local images are built identically to the official image except for retrieving the Solr artifacts locally instead of from the official release. | ||
This can be useful for testing out local changes as well as creating custom images for yourself or your organization. | ||
The task will output the image name to use at the end of the build. | ||
|
||
`./gradlew docker` | ||
|
||
For more info on building an image, run: | ||
|
||
`./gradlew helpDocker` | ||
|
||
### Running Solr on Kubernetes | ||
|
||
Solr has official support for running on Kubernetes, in the official Docker image. | ||
Please refer to the [Solr Operator](https://solr.apache.org/operator) home for details, tutorials and instructions. | ||
|
||
## Building Solr from Source | ||
Download the Java 11 JDK (Java Development Kit) or later. We recommend the OpenJDK | ||
distribution Eclipse Temurin available from https://adoptium.net/. | ||
You will need the JDK installed, and the $JAVA_HOME/bin (Windows: %JAVA_HOME%\bin) | ||
folder included on your command path. To test this, issue a "java -version" command | ||
from your shell (command prompt) and verify that the Java version is 11 or later. | ||
|
||
Download the Apache Solr distribution, from https://solr.apache.org/downloads.html. | ||
Unzip the distribution to a folder of your choice, e.g. C:\solr or ~/solr | ||
Alternately, you can obtain a copy of the latest Apache Solr source code | ||
directly from the Git repository: | ||
|
||
<https://solr.apache.org/community.html#version-control> | ||
|
||
Solr uses [Gradle](https://gradle.org/) as the build | ||
system. Navigate to the root of your source tree folder and issue the `./gradlew tasks` | ||
command to see the available options for building, testing, and packaging Solr. | ||
|
||
`./gradlew dev` will create a Solr executable suitable for development. | ||
cd to `./solr/packaging/build/dev` and run the `bin/solr` script | ||
to start Solr. | ||
|
||
NOTE: `gradlew` is the "Gradle Wrapper" and will automatically download and | ||
start using the correct version of Gradle for Solr. | ||
|
||
NOTE: `./gradlew help` will print a list of high-level tasks. There are also a | ||
number of plain-text files in <source folder root>/help. | ||
|
||
The first time you run Gradle, it will create a file "gradle.properties" that | ||
contains machine-specific settings. Normally you can use this file as-is, but it | ||
can be modified if necessary. | ||
|
||
Note as well that the gradle build does not create or copy binaries throughout the | ||
source repository so you need to switch to the packaging output folder `./solr/packaging/build`; | ||
the rest of the instructions below remain identical. The packaging directory | ||
is rewritten on each build. | ||
|
||
If you want to build the documentation, type `./gradlew -p solr documentation`. | ||
|
||
`./gradlew check` will assemble Solr and run all validation tasks unit tests. | ||
|
||
To build the final Solr artifacts run `./gradlew assemble`. | ||
|
||
Lastly, there is developer oriented documentation in `./dev-docs/README.adoc` that | ||
you may find useful in working with Solr. | ||
|
||
|
||
### Gradle build and IDE support | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. General about building from source and gradle - perhaps we should make some really good and thorough guides about that topic in But i see that the main focus here is merging the two READMEs, so feel free to postpone dev-docs work to later. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I agree with the sentiment! We should rationalize more the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That can be done in followup issues. I think we plan for a separate dev-guide, and not mash everything into ref-guide. |
||
|
||
- *IntelliJ* - IntelliJ idea can import the project out of the box. | ||
Code formatting conventions should be manually adjusted. | ||
- *Eclipse* - Not tested. | ||
- *Netbeans* - Not tested. | ||
|
||
## Contributing | ||
|
||
Please review the [Contributing to Solr Guide](https://cwiki.apache.org/confluence/display/solr/HowToContribute) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Later, the result of https://issues.apache.org/jira/browse/SOLR-15682 will be linked here... |
||
for information on contributing. | ||
|
||
## Discussion and Support | ||
|
||
- [Mailing Lists](https://solr.apache.org/community.html#mailing-lists-chat) | ||
- [Issue Tracker (JIRA)](https://issues.apache.org/jira/browse/SOLR) | ||
- IRC: `#solr` and `#solr-dev` on libera.chat | ||
- [Slack](https://solr.apache.org/community.html#slack) | ||
|
||
## Export control | ||
|
||
This distribution includes cryptographic software. The country in | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [Q] Is there a legal requirement to have this here, or is it here for practical purposes? (i.e. This mostly looks like legal boilerplate to me, but I'm not an expert so maybe there's actual nuggets in here useful to sysadmin/security folks?) Assuming this is here for legal reasons: looking at other projects (e.g. Elasticsearch, Vespa, Zookeeper) who use cryptography similar to Solr, none of them mention export controls in their top level repositories. Maybe we're the rare case of a project actually doing what's legally required, but I suspect we actually don't need this here. (Also, |
||
which you currently reside may have restrictions on the import, | ||
possession, use, and/or re-export to another country, of | ||
encryption software. BEFORE using any encryption software, please | ||
check your country's laws, regulations and policies concerning the | ||
import, possession, or use, and re-export of encryption software, to | ||
see if this is permitted. See <https://www.wassenaar.org/> for more | ||
information. | ||
|
||
The U.S. Government Department of Commerce, Bureau of Industry and | ||
Security (BIS), has classified this software as Export Commodity | ||
Control Number (ECCN) 5D002.C.1, which includes information security | ||
software using or performing cryptographic functions with asymmetric | ||
algorithms. The form and manner of this Apache Software Foundation | ||
distribution makes it eligible for export under the License Exception | ||
ENC Technology Software Unrestricted (TSU) exception (see the BIS | ||
Export Administration Regulations, Section 740.13) for both object | ||
code and source code. | ||
|
||
The following provides more details on the included cryptographic | ||
software: | ||
|
||
Apache Solr uses Apache Tika which uses the Bouncy Castle generic encryption libraries for | ||
extracting text content and metadata from encrypted PDF files. | ||
See https://www.bouncycastle.org/ for more details on Bouncy Castle. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[0] It feels weird to single out enterprise search here as being our main (only?) use case. We're also an ecommerce search engine, and an analytics engine, etc.
(I understand this language is probably carried over from one of the existing READMEs, and used in many places...no need to fix it here necessarily if you're not interested.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I completely agree with you @gerlowskija on this... It IS front and center on the website at https://solr.apache.org/ ;-(. Worth a seperate ticket?
???
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, we should fix it everywhere. I changed the Official docker image description to remove it. But we should standardize around a common language, and set that everywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@HoustonPutman you want to open a ticket? If you get the consensus, I'd volunteer to do the updates ;-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://issues.apache.org/jira/browse/SOLR-16295
We can brainstorm there