iop
/
superset
4
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[releases] New, email automation for apache releases (#8612)

This commit is contained in:
Daniel Vaz Gaspar 2019-11-25 10:01:31 +00:00 committed by GitHub
parent 4a34c553c7
commit f04b72c341
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 631 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
RELEASING/README.md
View File

@ -119,7 +119,7 @@ release on Superset's repo (MAJOR.MINOR branch), the following script will clone
the tag and create a signed source tarball from it:
```bash
# make_tarball you use the previouly set environment variables
# make_tarball will use the previously set environment variables
# you can override by passing arguments: make_tarball.sh <SUPERSET_VERSION> <SUPERSET_VERSION_RC> "<PGP_KEY_FULLNAME>"
./make_tarball.sh
```
@ -166,17 +166,65 @@ Now you're ready to start the [VOTE] thread. Here's an example of a
previous release vote thread:
https://lists.apache.org/thread.html/e60f080ebdda26896214f7d3d5be1ccadfab95d48fbe813252762879@<dev.superset.apache.org>
To easily send a voting request to Superset community, still on the `superset/RELEASING` directory:
```bash
# Note use Superset's virtualenv
(venv)$ python send_email.py vote_pmc
```
The script will interactively ask for extra information so it can authenticate on the Apache Email Relay.
The release version and release candidate number are fetched from the previously set environment variables.
```bash
Sender email (ex: user@apache.org): your_apache_email@apache.org
Apache username: your_apache_user
Apache password: your_apache_password
```
Once 3+ binding votes (by PMC members) have been cast and at
least 72 hours have past, you can post a [RESULT] thread:
https://lists.apache.org/thread.html/50a6b134d66b86b237d5d7bc89df1b567246d125a71394d78b45f9a8@%3Cdev.superset.apache.org%3E
To easily send the result email, still on the `superset/RELEASING` directory:
```bash
# Note use Superset's virtualenv
(venv)$ python send_email.py result_pmc
```
The script will interactively ask for extra information needed to fill out the email template. Based on the
voting description, it will generate a passing, non passing or non conclusive email.
here's an example:
```bash
Sender email (ex: user@apache.org): your_apache_email@apache.org
Apache username: your_apache_user
Apache password: your_apache_password
A List of people with +1 binding vote (ex: Max,Grace,Krist): Daniel,Alan,Max,Grace
A List of people with +1 non binding vote (ex: Ville): Ville
A List of people with -1 vote (ex: John):
```
Following the result thread, yet another [VOTE] thread should be
started at general@incubator.apache.org.
To easily send the voting request to Apache community, still on the `superset/RELEASING` directory:
```bash
# Note use Superset's virtualenv
(venv)$ python send_email.py vote_ipmc
```
### Announcing
Once it's all done, an [ANNOUNCE] thread announcing the release to the dev@ mailing list is the final step.
```bash
# Note use Superset's virtualenv
(venv)$ python send_email.py announce
```
### Validating a release
https://www.apache.org/info/verification.html

57
RELEASING/email_templates/announce.j2 Normal file
View File

@ -0,0 +1,57 @@
{#
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-#}
To: {{ receiver_email }}
From: {{ sender_email }}
Subject: [ANNOUNCE] Apache {{ project_name }} (Incubating) version {{ version }} Released
Hello Community,
The Apache {{ project_name }} (incubating) team is pleased to announce that {{ project_name }}
{{ version }} has just been released.
{{ project_description }}
The official source release:
https://www.apache.org/dist/incubator/{{ project_module }}/{{ version }}
The Pypi package:
https://pypi.org/project/apache-superset/
If you have any usage questions, or have problems when upgrading or
find any problems about enhancements included in this release, please
don’t hesitate to let us know by sending feedback to this mailing
list.
=====
*Disclaimer*
Apache {{ project_name }} is an effort undergoing incubation at The Apache Software
Foundation (ASF), sponsored by the Incubator. Incubation is required of all
newly accepted projects until a further review indicates that the
infrastructure, communications, and decision making process have stabilized
in a manner consistent with other successful ASF projects. While incubation
status is not necessarily a reflection of the completeness or stability of
the code, it does indicate that the project has yet to be fully endorsed by
the ASF.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

58
RELEASING/email_templates/result_ipmc.j2 Normal file
View File

@ -0,0 +1,58 @@
{#
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-#}
To: {{ receiver_email }}
From: {{ sender_email }}
Subject: [RESULT] [VOTE] Release Apache {{ project_name }} (incubating) version {{ version }}
Thanks to everyone that participated. The vote to release
Apache {{ project_name }} (incubating) version {{ version }} is now closed.
{% if vote_negatives|length > 0 %}
The vote did not PASS with {{vote_bindings|length}} binding +1, {{ vote_nonbindings|length}} non binding +1 and {{vote_negatives|length}} -1 votes:
{% elif vote_bindings|length > 2 %}
The vote PASSED with {{vote_bindings|length}} binding +1, {{ vote_nonbindings|length}} non binding +1 and {{vote_negatives|length}} -1 votes:
{% else %}
The vote is non conclusive with {{vote_bindings|length}} binding +1, {{ vote_nonbindings|length}} non binding -1 and {{vote_negatives|length}} -1 votes:
{% endif %}
{% if vote_bindings|length > 0 -%}
Binding votes:
{% for voter in vote_bindings -%}
- {{ voter }}
{% endfor %}
{%- endif %}
{% if vote_nonbindings|length > 0 -%}
Non binding votes:
{% for voter in vote_nonbindings -%}
- {{ voter }}
{% endfor %}
{%- endif %}
{% if vote_negatives|length > 0 -%}
Negative votes:
{% for voter in vote_negatives -%}
- {{ voter }}
{% endfor %}
{%- endif %}
We will work to complete the release process.
Thanks,
The Apache {{ project_name }} (Incubating) Team

58
RELEASING/email_templates/result_pmc.j2 Normal file
View File

@ -0,0 +1,58 @@
{#
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-#}
To: {{ receiver_email }}
From: {{ sender_email }}
Subject: [RESULT] [VOTE] Release Apache {{ project_name }} (incubating) {{ version }} based on Superset {{ version_rc }}
Thanks to everyone that participated. The vote to release
Apache {{ project_name }} (incubating) version {{ version }} based on {{ version_rc }} is now closed.
{% if vote_negatives|length > 0 -%}
The vote did NOT PASS with {{vote_bindings|length}} binding +1, {{ vote_nonbindings|length}} non binding +1 and {{vote_negatives|length}} -1 votes:
{% elif vote_bindings|length > 2 -%}
The vote PASSED with {{vote_bindings|length}} binding +1, {{ vote_nonbindings|length}} non binding +1 and {{vote_negatives|length}} -1 votes:
{% else -%}
The vote is non conclusive with {{vote_bindings|length}} binding +1, {{ vote_nonbindings|length}} non binding -1 and {{vote_negatives|length}} -1 votes:
{%- endif %}
{% if vote_bindings|length > 0 -%}
Binding votes:
{% for voter in vote_bindings -%}
- {{ voter }}
{% endfor -%}
{%- endif %}
{% if vote_nonbindings|length > 0 -%}
Non binding votes:
{% for voter in vote_nonbindings -%}
- {{ voter }}
{% endfor -%}
{%- endif %}
{% if vote_negatives|length > 0 -%}
Negative votes:
{% for voter in vote_negatives -%}
- {{ voter }}
{% endfor -%}
{%- endif %}
We will work to complete the release process.
Thanks,
The Apache {{ project_name }} (Incubating) Team

63
RELEASING/email_templates/vote_ipmc.j2 Normal file
View File

@ -0,0 +1,63 @@
{#
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-#}
To: {{ receiver_email }}
From: {{ sender_email }}
Subject: [VOTE] Release Apache {{ project_name }} (incubating) version {{ version }}
Hello IPMC,
The Apache {{ project_name }} (incubating) community has voted on and approved a proposal to
release Apache {{ project_name }} (incubating) version {{ version }}.
The voting thread can be found here: {{ voting_thread }}
{% if vote_mentors|length > 0 %}
Here are the binding +1 votes from mentors, carrying over from the podling vote:
{% for voter in vote_mentors -%}
- {{ voter }}
{% endfor %}
{% endif -%}
We now kindly request the Incubator PMC members review and vote on this
incubator release.
{{ project_description }}
The release candidate:
https://dist.apache.org/repos/dist/dev/incubator/{{ project_module }}/{{ version_rc }}/
Git tag for the release:
https://github.com/apache/incubator-{{ project_module }}/tree/{{ version_rc }}
The Change Log for the release:
https://github.com/apache/incubator-{{ project_module }}/blob/{{ version_rc }}/CHANGELOG.md
public keys are available at:
https://www.apache.org/dist/incubator/{{ project_module }}/KEYS
The vote will be open for at least 72 hours or until the necessary number
of votes are reached.
Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason
Thanks,
The Apache {{ project_name }} (Incubating) Team

51
RELEASING/email_templates/vote_pmc.j2 Normal file
View File

@ -0,0 +1,51 @@
{#
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-#}
To: {{ receiver_email }}
From: {{ sender_email }}
Subject: [VOTE] Release Apache {{ project_name }} (incubating) {{ version }} based on Superset {{ version_rc }}
Hello {{ project_name }} Community,
This is a call for the vote to release Apache {{ project_name }}
(incubating) version {{ version }}.
The release candidate:
https://dist.apache.org/repos/dist/dev/incubator/{{ project_module }}/{{ version_rc }}/
Git tag for the release:
https://github.com/apache/incubator-{{ project_module }}/tree/{{ version_rc }}
The Change Log for the release:
https://github.com/apache/incubator-{{ project_module }}/blob/{{ version_rc }}/CHANGELOG.md
public keys are available at:
https://www.apache.org/dist/incubator/{{ project_module }}/KEYS
The vote will be open for at least 72 hours or until the necessary number
of votes are reached.
Please vote accordingly:
[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason
Thanks,
The Apache {{ project_name }} (Incubating) Team

295
RELEASING/send_email.py Executable file
View File

@ -0,0 +1,295 @@
#!/usr/bin/python3
#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from typing import List
import smtplib
import ssl
try:
import jinja2
except ModuleNotFoundError:
exit("Jinja2 is a required dependency for this script")
try:
import click
except ModuleNotFoundError:
exit("Click is a required dependency for this script")
SMTP_PORT = 587
SMTP_SERVER = "mail-relay.apache.org"
PROJECT_NAME = "Superset"
PROJECT_MODULE = "superset"
PROJECT_DESCRIPTION = "Apache Superset (incubating) is a modern, enterprise-ready business intelligence web application"
def string_comma_to_list(message: str) -> List[str]:
if not message:
return []
return message.split(",")
def send_email(
smtp_server: str,
smpt_port: int,
username: str,
password: str,
sender_email: str,
receiver_email: str,
message: str,
):
"""
Send a simple text email (SMTP)
"""
context = ssl.create_default_context()
with smtplib.SMTP(smtp_server, smpt_port) as server:
server.starttls(context=context)
server.login(username, password)
server.sendmail(sender_email, receiver_email, message)
def render_template(template_file: str, **kwargs) -> str:
"""
Simple render template based on named parameters
:param template_file: The template file location
:kwargs: Named parameters to use when rendering the template
:return: Rendered template
"""
template = jinja2.Template(open(template_file).read())
return template.render(kwargs)
def inter_send_email(username, password, sender_email, receiver_email, message):
print("--------------------------")
print("SMTP Message")
print("--------------------------")
print(message)
print("--------------------------")
confirm = input("Is the Email message ok? (yes/no): ")
if confirm not in ("Yes", "yes", "y"):
exit("Exit by user request")
try:
send_email(
SMTP_SERVER,
SMTP_PORT,
username,
password,
sender_email,
receiver_email,
message,
)
print("Email sent successfully")
except smtplib.SMTPAuthenticationError:
exit("SMTP User authentication error, Email not sent!")
except Exception as e:
exit(f"SMTP exception {e}")
class BaseParameters(object):
def __init__(
self, email=None, username=None, password=None, version=None, version_rc=None
):
self.email = email
self.username = username
self.password = password
self.version = version
self.version_rc = version_rc
self.template_arguments = dict()
def __repr__(self):
return f"Apache Credentials: {self.email}/{self.username}/{self.version}/{self.version_rc}"
@click.group()
@click.pass_context
@click.option(
"--apache_email",
prompt="Apache Email",
help="Your Apache email this will be used for SMTP From",
)
@click.option(
"--apache_username",
prompt="Apache username",
help="Your LDAP Apache username",
)
@click.option(
"--apache_password",
prompt="Apache password",
hide_input=True,
help="Your LDAP Apache password",
)
@click.option("--version", envvar="SUPERSET_VERSION")
@click.option("--version_rc", envvar="SUPERSET_VERSION_RC")
def cli(ctx, apache_email, apache_username, apache_password, version, version_rc):
""" Welcome to releasing send email CLI interface! """
base_parameters = BaseParameters(
apache_email, apache_username, apache_password, version, version_rc
)
base_parameters.template_arguments["project_name"] = PROJECT_NAME
base_parameters.template_arguments["project_module"] = PROJECT_MODULE
base_parameters.template_arguments["project_description"] = PROJECT_DESCRIPTION
base_parameters.template_arguments["version"] = base_parameters.version
base_parameters.template_arguments["version_rc"] = base_parameters.version_rc
base_parameters.template_arguments["sender_email"] = base_parameters.email
ctx.obj = base_parameters
@cli.command("vote_pmc")
@click.option(
"--receiver_email",
default="dev@superset.apache.org",
type=str,
prompt="The receiver email (To:)",
)
@click.pass_obj
def vote_pmc(base_parameters, receiver_email):
template_file = "email_templates/vote_pmc.j2"
base_parameters.template_arguments["receiver_email"] = receiver_email
message = render_template(template_file, **base_parameters.template_arguments)
inter_send_email(
base_parameters.username,
base_parameters.password,
base_parameters.template_arguments["sender_email"],
base_parameters.template_arguments["receiver_email"],
message,
)
@cli.command("result_pmc")
@click.option(
"--receiver_email",
default="dev@superset.apache.org",
type=str,
prompt="The receiver email (To:)",
)
@click.option(
"--vote_bindings",
default="",
type=str,
prompt="A List of people with +1 binding vote (ex: Max,Grace,Krist)",
)
@click.option(
"--vote_nonbindings",
default="",
type=str,
prompt="A List of people with +1 non binding vote (ex: Ville)",
)
@click.option(
"--vote_negatives",
default="",
type=str,
prompt="A List of people with -1 vote (ex: John)",
)
@click.pass_obj
def result_pmc(base_parameters, receiver_email, vote_bindings, vote_nonbindings, vote_negatives):
template_file = "email_templates/result_pmc.j2"
base_parameters.template_arguments["receiver_email"] = receiver_email
base_parameters.template_arguments["vote_bindings"] = string_comma_to_list(
vote_bindings
)
base_parameters.template_arguments["vote_nonbindings"] = string_comma_to_list(
vote_nonbindings
)
base_parameters.template_arguments["vote_negatives"] = string_comma_to_list(
vote_negatives
)
message = render_template(template_file, **base_parameters.template_arguments)
inter_send_email(
base_parameters.username,
base_parameters.password,
base_parameters.template_arguments["sender_email"],
base_parameters.template_arguments["receiver_email"],
message,
)
@cli.command("vote_ipmc")
@click.option(
"--receiver_email",
default="general@incubator.apache.org",
type=str,
prompt="The receiver email (To:)",
)
@click.option("--voting_thread", prompt="The URL for the PMC voting thread")
@click.option(
"--vote_mentors",
default="",
type=str,
prompt="A list of mentors that have already voted (ex: Alan,Justin)",
)
@click.pass_obj
def vote_ipmc(base_parameters, receiver_email, voting_thread, vote_mentors):
template_file = "email_templates/vote_ipmc.j2"
base_parameters.template_arguments["receiver_email"] = receiver_email
base_parameters.template_arguments["voting_thread"] = voting_thread
base_parameters.template_arguments["vote_mentors"] = string_comma_to_list(
vote_mentors
)
message = render_template(template_file, **base_parameters.template_arguments)
inter_send_email(
base_parameters.username,
base_parameters.password,
base_parameters.template_arguments["sender_email"],
base_parameters.template_arguments["receiver_email"],
message,
)
@cli.command("result_ipmc")
@click.option(
"--receiver_email",
default="general@incubator.apache.org",
type=str,
prompt="The receiver email (To:)",
)
@click.pass_obj
def result_ipmc(base_parameters, receiver_email):
template_file = "email_templates/result_ipmc.j2"
base_parameters.template_arguments["receiver_email"] = receiver_email
message = render_template(template_file, **base_parameters.template_arguments)
inter_send_email(
base_parameters.username,
base_parameters.password,
base_parameters.template_arguments["sender_email"],
base_parameters.template_arguments["receiver_email"],
message,
)
@cli.command("announce")
@click.option(
"--receiver_email",
default="dev@superset.apache.org",
type=str,
prompt="The receiver email (To:)",
)
@click.pass_obj
def announce(base_parameters, receiver_email):
template_file = "email_templates/announce.j2"
base_parameters.template_arguments["receiver_email"] = receiver_email
message = render_template(template_file, **base_parameters.template_arguments)
inter_send_email(
base_parameters.username,
base_parameters.password,
base_parameters.template_arguments["sender_email"],
base_parameters.template_arguments["receiver_email"],
message,
)
cli()