Improve Server CE dev environment (#13820)

* Avoid stray backslash in ggrep pattern
* Add new services to README
* Avoid building in bin/up
* Add init script
* Update README.md
* Use cURL to wait for MongoDB
* Add a note about COMPOSE_PARALLEL_LIMIT
* Use the new callout syntax

GitOrigin-RevId: 9aaec37e893a2da807fed2dc4b00fa52a433c390

develop/README.md

@@ -2,7 +2,22 @@
 
 ## Building and running
 
-In this `develop` directory, build and start the services:
+In this `develop` directory, build the services:
+
+```shell
+bin/build
+```
+
+> [!NOTE]
+> If Docker is running out of RAM while building the services in parallel, create a `.env` file in this directory containing `COMPOSE_PARALLEL_LIMIT=1`.
+
+Next, initialize the database:
+
+```shell
+bin/init
+```
+
+Then start the services:
 
 ```shell
 bin/up
@@ -10,8 +25,6 @@ bin/up
 
 Once the services are running, open <http://localhost/launchpad> to create the first admin account.
 
-After making any changes to the code, run `bin/up` manually to rebuild the changed Docker images and recreate the changed containers.
-
 ## TeX Live
 
 Compiling a PDF requires building a TeX Live image to handle the compilation inside Docker:
@@ -20,35 +33,37 @@ Compiling a PDF requires building a TeX Live image to handle the compilation ins
 docker build texlive -t texlive-full
 ```
 
-> **Note**
-> To compile on a macOS host, you'll need to override the path to the Docker socket by creating a `.env` file in this directory, containing
+> [!NOTE]
+> To compile on a macOS host, you may need to override the path to the Docker socket by creating a `.env` file in this directory, containing
 > `DOCKER_SOCKET_PATH=/var/run/docker.sock.raw`
 
 ## Development
 
-To avoid running `bin/up` after every code change, you can run Overleaf
-Community Edition in _development mode_, where services will automatically
-update on code changes. To do this, use the included `bin/dev` script:
+To avoid running `bin/build && bin/up` after every code change, you can run Overleaf
+Community Edition in _development mode_, where services will automatically update on code changes.
+
+To do this, use the included `bin/dev` script:
 
 ```shell
 bin/dev
 ```
 
-This will start all services using `nodemon`, which will automatically monitor
-the code and restart the services as necessary. This will incur a performance
-hit, especially on macOS, so in order to only start a subset of the services,
-provide a space-separated list to the `bin/dev` script.
+This will start all services using `nodemon`, which will automatically monitor the code and restart the services as necessary.
+
+To improve performance, you can start only a subset of the services in development mode by providing a space-separated list to the `bin/dev` script:
 
 ```shell
 bin/dev [service1] [service2] ... [serviceN]
 ```
 
-> **Note**
+> [!NOTE]
 > Starting the `web` service in _development mode_ will only update the `web`
 > service when backend code changes. In order to automatically update frontend
 > code as well, make sure to start the `webpack` service in _development mode_
 > as well.
 
+If no services are named, all services will start in development mode.
+
 ## Debugging
 
 When run in _development mode_ most services expose a debugging port to which
@@ -68,6 +83,8 @@ each service:
 | `filestore`        | 9235 |
 | `notifications`    | 9236 |
 | `real-time`        | 9237 |
+| `history-v1`       | 9239 |
+| `project-history`  | 9240 |
 
 To attach to a service using Chrome's _remote debugging_, go to
 <chrome://inspect/> and make sure _Discover network targets_ is checked. Next

develop/bin/build

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+docker-compose build --pull "$@"

develop/bin/down

@@ -1,3 +1,3 @@
 #!/usr/bin/env bash
 
-docker-compose down
+docker-compose down "$@"

develop/bin/init

@@ -0,0 +1,6 @@
+#!/usr/bin/env bash
+
+docker-compose up --detach mongo
+curl --max-time 10 --retry 5 --retry-delay 5 --retry-all-errors --silent --output /dev/null localhost:27017
+docker-compose exec mongo mongosh --eval "rs.initiate({ _id: 'overleaf', members: [{ _id: 0, host: 'mongo:27017' }] })"
+docker-compose down mongo

develop/bin/logs

@@ -5,5 +5,5 @@ docker-compose logs --follow --tail 10 --no-color "$@" \
   | ggrep --line-buffered --invert-match "health.check" \
   | ggrep --line-buffered --invert-match "slow event loop" \
   | ggrep --line-buffered --invert-match "process.memoryUsage" \
-  | ggrep --line-buffered --only-matching "\{.*" \
+  | ggrep --line-buffered --only-matching "[{].*" \
   | bunyan --output short

develop/bin/up

@@ -1,3 +1,3 @@
 #!/usr/bin/env bash
 
-docker-compose up --build --detach "$@"
+docker-compose up --detach "$@"

develop/dev.env

@@ -8,7 +8,7 @@ GRACEFUL_SHUTDOWN_DELAY=0
 HISTORY_V1_HOST=history-v1
 LISTEN_ADDRESS=0.0.0.0
 MONGO_HOST=mongo
-MONGO_URL=mongodb://mongo/sharelatex?readPreference=secondary&directConnection=true
+MONGO_URL=mongodb://mongo/sharelatex?directConnection=true
 NOTIFICATIONS_HOST=notifications
 PROJECT_HISTORY_HOST=project-history
 REALTIME_HOST=real-time

develop/docker-compose.yml

@@ -95,10 +95,6 @@ services:
       - "127.0.0.1:27017:27017" # for debugging
     volumes:
       - mongo-data:/data/db
-    healthcheck:
-      test: 'test $$(mongosh --eval "rs.initiate({ _id: "overleaf", members: [ { _id: 0, host: "mongo:27017" } ] }).ok || rs.status().ok" --quiet) -eq 1'
-      interval: 30s
-      start_period: 30s
 
   notifications:
     build: